Page MenuHomeSoftware Heritage
Files F9124046

api-backend-protocol.txt
No OneTemporary
Actions

api-backend-protocol.txt
View Options

Design considerations
=====================
# Goal
Load the representation of a git,svn, csv, tarball, et al. repository in software
heritage's backend.
# Nomenclature
## Infrastructure
- swh: software heritage
- worker: an instance in charge of loading a repository in swh's backend
- backend: the software heritage storage mechanism used (at the moment,
postgresql db + file storage) and by extension its api.
- backend api: internal private api used to by worker and backend to
communicate. Can be used in `backend`'s stead.
## Object
- revision: A snapshot of a software project at a specific point in time. Ex: git commit
- directory: A file-system directory. Ex: git tree
- content: Checksums about actual file content. Ex: git blob's checksum.
- release: A "memorable" point in the development history of a project. Ex: git's annotated tag
# Scenario
In the following, we will describe with different granularities what will
happen between workers and backend.
## 1
A worker parses a repository.
It sends the parsing result to the backend.
First, the worker sends a complete list of all sha1s encountered.
The server responds with a complete list of unknowns sha1s.
The worker sends those sha1s and their associated data to the server.
The server store what it receives.
## 2
1. Worker parses local repository and build a memory model of the repository.
2. Worker sends repository's list of all encountered sha1s to the backend.
3. Backend replies with unknown sha1.
4. Worker sends all blob's data and metadata through 1 (or more) request(s).
5. Backend stores them and finish the transaction.
6. Worker sends all trees' data and metadata through 1 (or more) request(s).
7. Backend stores them and finish the transaction.
8. Worker sends all commits' data and metadata through 1 (or more) request(s).
9. Backend stores them and finish the transaction.
10. Worker is done.

File Metadata

Mime Type
text/plain
Expires
Sat, Jun 21, 6:36 PM (1 w, 6 d ago)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
3403489

Event Timeline

About · Developer information · Legal