We've created a compressed representation of the archive Merkle DAG, from snapshot nodes down to contents.
Strictly speaking that is exhaustive, but for some use cases (e.g., list all the different origins where a given node has been found) it would be useful to also have nodes representing origins.
Now that we know compressing the full Merkle DAG is doable, we should give a try to the extended version that also includes origin nodes. It shouldn't be much harder, given the added nodes/edges are not that numerous.
Description
Description
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Migrated | gitlab-migration | T1867 compress Merkle DAG and origin nodes together | ||
| Migrated | gitlab-migration | T1731 Intrinsic identifiers for origins | ||
| Migrated | gitlab-migration | T1915 Add support for origin nodes in graph service API |
Event Timeline
Comment Actions
Due to multiple server maintenance, the process was re-started a few times, but it is now finished and results are uploaded in the annex: https://annex.softwareheritage.org/public/dataset/graph/latest/compressed/all+ori/