LGTM, just a couple of wishes (and we are not even close to xmas!)

it's unclear to me if the hunk in swh_storage_backend_config is related with the fix or not.
isn't the very last diff hunk enough for the job?

rebase

fix docstring (thx ardumont)

several fixes and add a --template option to 'swh db create'

better ci mesg

In D3997#98782, @douardda wrote:

In D3997#98758, @douardda wrote:

I don't understand how this is supposed to work.

How does the flavor "name" is taken in consideration? Is it supposed to select "flavored" versions of sql files? What's the pattern ? As written, I don't understand how the flavor "name" is used besides being written the dbflavor table.

Oh now I get it (after reading D3981)

It's the presence of a xxx-flavor.sql that triggers the insertion/update of the flavor given as cli argument in the dbflavor table, which then specify how the other sql files are executed (thx to /if statements in there).

In D3997#98758, @douardda wrote:

I don't understand how this is supposed to work.

How does the flavor "name" is taken in consideration? Is it supposed to select "flavored" versions of sql files? What's the pattern ? As written, I don't understand how the flavor "name" is used besides being written the dbflavor table.

typo (thx anlambert)

what about the deduplication flavor? it's not described in the ci message. Also this description of flavors should be documented somewhere (I mean somewhere else than the ci message)

I don't understand how this is supposed to work.

I'm not sure I understand what's a stake here, but it really looks odd.

just my 2c. do we want to allow deposit update in the ready state or do we want to be able to get back from ready to partial?

typos (thx ardumont) + better ci message (thx vlorentz)

In D3975#98153, @vlorentz wrote:

Why do we need this?

because of

slightly better comments

In D3975#98089, @olasd wrote:

It took me a bit to understand what the sorteds were achieving (dedup the list while keeping it in the same order), so I wouldn't mind if you turned them into an explicit loop, or at least if you added a comment.

fix db/test_cli

rebase

rebase

FTR, in a test setup I made a few days ago on docker, I had a git loader crunching ~28GB of RES mem (on 32 available on that machine). Not sure which repo it was ingesting, but it was on codeberg.

FTR, in a previous life, I've set up a json web token auth validation in varnish.

In D3900#96997, @anlambert wrote:

Almost good but there is still something that puzzles me regarding global configuration handling. Let me explain.

What we want to achieve here is to share Web API URL and authentication token across several swh components (swh-scanner and swh-web-client at the moment)
without duplicating it in multiple configuration files. So basically we want to put that configuration in the global.yml file.

nitpicking: the config management part does need to be in the same git revision as the auth support.

LGTM, as far as I can tell (but I cannot tell that much TBH)

Just as small thing: please use imperative mood in the commit message. The subject is ok, but the "moved [...]" is not.
Also it would make more sense to explicitly refers to cli.py instead of just cli in the subject, i.e.:

Let's consider this done for now. Still not "great" loading time, but much better than before.

fix db/test_cli

forgot setup.py

also fix test_cli

rename as objstorage_cli_group

In D3918#96730, @vlorentz wrote:

What about objstorage_cli instead?

fix test_cli

Let's call it fixed (until further notice).

rebase + fix test_cli

not reviewed the test yet

pfff

fix a doctest

use string annotations

same *with* the file in the commit --amend

use strin annotations as suggested by vlorent

use string annotation instead of comment one, as suggested by vlorentz