In T3627#73323, @zack wrote:

Thanks for the summaries @olasd, both here and on list.
I've followed up on list.

Meanwhile here's what I propose we do (spoiler!):

a) A4: add to the archive Merkle DAG only the filtered snapshot (referencing "intrinsic" branches only, as per A2) and its transitive closure

Thanks for the summaries @olasd, both here and on list.
I've followed up on list.

Related to T3627

Sent a summary of this discussion to the swh-devel list for input:

In T3627#72544, @douardda wrote:

B3 I am not convinced a "synthetic" flag on the Snapshot branch makes sense, or at least I find this name confusing, especially considering we already have a synthetic flag on Revision: it's not synthetic in the sense of it's not object crafted by SWH, it comes from the origin.

B3 I am not convinced a "synthetic" flag on the Snapshot branch makes sense, or at least I find this name confusing, especially considering we already have a synthetic flag on Revision: it's not synthetic in the sense of it's not object crafted by SWH, it comes from the origin.

I would like us to conclude this discussion soon.

Now, I still don't understand what mapping is to be stored in the extid table. What is
meaning of (version 0, sha1-git of the commit/tag, revision/release id) above? (I
expect a mapping to be a couple).

In T3635#72206, @douardda wrote:

Then I don't really get how this can help if we don't load revisions in topological order.

Ok I think what puzzle me in this description is the fact the 2 first bullets of the "git loader adaptations" are actually only one point: at the end of a successful loading, store a mapping in the extid table.

(I've removed T3653 as parent as this is a somewhat longer term endeavour. Not the topological sorting itself, but making sure that (most) existing revisions aren't dangling, before we can use this topological guarantee)

Actually deployed and the number of oom actually decreased.

Deploy storage v0.38 on worker (proxy buffer/filter adaptations client/loader side).
Restarted all loaders with it.

Issues related to git dumb loading have been handled and the 5 dumb origins that were failing have been resubmitted through save code now and successfully loaded, closing this.

In T3627#71819, @vlorentz wrote:

An alternative to annotating synthetic refs: add a "type" or "forge_type" attribute to snapshots.

btw ^

In T3625#71799, @olasd wrote:

While we're at it, we should probably be adding some thresholds in the buffer proxy for:

• cumulated length of messages for revisions and releases

I concur with this analysis btw

An alternative to annotating synthetic refs: add a "type" or "forge_type" attribute to snapshots.