My API idea was to simply have something like ENTRIES_REGEXP = r'^.*:.*$' as a class attribute of each type of directory, and a validate_entry(self, name: str) method which, by default, just checks that it matches the regexp.

We also need to discuss what exactly we put in cache/. I thought about symlinks to archive/ and meta/, what do you think? Removing the symlinks also means removing the data from the cache.

I think I understand what your fill_direntry_cache function is trying to do: you want to avoid fetching the history multiple times by doing the request only once and writing the direntry cache of all the children recursively?
Would it be maybe better to instead have a small LRU cache for the API queries, and keep the direntry code simple and fully lazy?

Looks good apart from two small things.

One thing I don't really like here is that FuseEntries cannot easily list their own entries easily using the cache when available. I would much rather have the cache logic moved inside FuseEntry like what we discussed.

lookup() should ideally be O(1).

This is looking pretty great. I see three more good refactoring possibilities:

Fix BVGraph/ImmutableGraph implicit naming

Yes, the correct method name is enter_context

I would do a bit of refactoring:

No, only the edge part is done, we still need a parquet and a CSV exporter :/

It is already running on granet :-)

We considered three possibilities for the schema (assuming that we want to get rid of the three separate tables for dir_entries, rev_entries and file_entries -- otherwise, there's 6 possibilities).

rebase

Fix useless dependency

Thanks, this looks good!

Already landed

Rebase + add commit message

• Remove debug prints

Obsoleted by D3011

Just to be clear, the problem here wasn't directly linked to swhgraphshm but simply to the amount of available memory, because the MAP_PRIVATE flag tried to reserve all that memory to be able to perform copy on write. Using MAP_SHARED + PROT_READ avoids having this memory reservation and fixes the issue. swhgraphshm was just a random process taking a lot of the available ram, not specifically the reason why it failed.

Fixed and added a test for that problem.