Thanks for the bug report. Do you have a log of what happens upon bin/update if pre-commit is missing? (it's sure is needed for committing, but i'm surprised it's needed for bin/update).
Also, is it needed for a raw "bin/update" or only for "bin/update -i" (that also install Python dependencies?).

oh, also, this diff should close T2647 in its commit message

note that this diff should close T2595, so the commit message should be updated

My answer is a very emphatic YES.

In T2912#55860, @rdicosmo wrote:

In T2912#55849, @olasd wrote:

I think we should be able to decouple these counters completely from the loaders, and have them directly updated/handled by a client of the swh-journal. This would be a "centralized" component, but which we can parallelize quite heavily thanks to basic kafka design. We can also leverage the way kafka clients do parallelism to sidestep the locking issues arising in a potentially distributed filter.

Maybe my writing was not all that clear: I also had in mind a single centralised component (the ArchiveCounter) per Bloom filter, receiving the lists newcontents of ids from the loaders.
Getting the feed of ids from swh-journal instead of from the loaders is really neat: we avoid touching the loader code, and we gain a better capability of monitoring the load on the ArchiveCounter, so I'm all for it :-)

I don't like this "asymmetric" approach much. I'd rather have both caches being equal and taking as class constructor input an optional sqlite connection. If it's None, they'll each create a connection (and not share it). If a connection is passed, they use it and do not close it on exit. Then, the init code in FuseCache takes care of initializing the shared sqlite connection, and passing it to both constructors. This will make the classes more reusable and the init code more clean.

address review comments

In T2917#55617, @zvr wrote:

Please, not only document but add to the code something like
python_requires='>=3.7'
to setup.py so that the Pypi packages are not installable in older versions.

Yeah, we really depend on recent-ish Python (and not only for f-strings, a lot of typing-related stuff also comes to mind).
3.5 is no longer supported (and I have my doubts about 3.6 too; we generally follow Debian stable, which is on Python 3.7 now).

conditionally accepted with very minor changes to be implemented before landing

• doc index: improve short descriptions of entry points
• mirror doc: full reread with improvements here and there

• mirror doc: full reread with improvements here and there

• mirror doc: merge all topics in a single list

• doc index: improve short descriptions of entry points
• mirror doc: full reread with improvements here and there

LGTM in general, I've just nitpicked about a couple of things.

conditionally accepted, please generalize the check before landing (see comment)

LGTM (but please fix the minor docstring point I've mentioned before landing)