I guess this also depends on a packagist loader, which we do not have at all for now...

rebase + anlambert's comment

In D3841#94881, @anlambert wrote:

While you are are on it, can you also remove the objstorage dependency in swh-web service definition inside docker-compose.yml ?

On the other hand, it will be computationally expensive, especially for contents.
What do you think?

Also beware that the default pagination value in the gitea lister is 3 (https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitea/lister.py$23) so it is very slow.

Ok I was expecting something a bit smart in explore.sapk.fr, but not really:

now we have the gitea lister, we should (upgrade swh.lister on prod and) add a few listing tasks, like this fsfe instance, as well as other instances like https://codeberg.org.

small improvements

remove the webapp->objstorage RPC link

In D3838#94866, @douardda wrote:

general-architecture.svg355 KBDownload

In D3838#94841, @anlambert wrote:

For the record, there is not really a RPC link between the webapp and the objstorage as its use is wrapped by the content_* methods of the storage interface.
So I think you should remove that new link.

improve the diagram

rebase

type annotate the new ensure_check_config helper function

rebase + add closes T2525 in the ci msg

rebase

rebase

rebase

rebase

rebase

pick the correct commit to be stashed...

rebase

rebase

fix failed black magic as ween by ardumont

rebase

rebase

ok but please explain why in your commit message...

as reported by vlorentz, duplicate of D3811

as said in the commit message, It must be updated jointly in 30-swh-schema.sql
(including after a rebase...)

Just to make is clear, my main objective here is to have a seatbelt for the mirror scenario: prevent a mirror with updated code but not-yet-updated database from even running until the DB migration script has been executed.

In D3800#94004, @vlorentz wrote:

Yeah but I didn't find a very satisfying solution either. Well, I guess I could do something like this:

CONVERTERS: Dict[str, Callable[[BaseDb, Dict[str, Any]], BaseModel]] = {
    **{
        type_: lambda db, obj: converter(obj)
        for (type_, converter) in object_converter_fn.items()
    },
    "directory": directory_converter,                          
    "raw_extrinsic_metadata": raw_extrinsic_metadata_converter,
    "revision": revision_converter,                            
    "release": release_converter,                              
    "snapshot": snapshot_converter,                            
}

But it's not much of an improvement

LGTM, but see my comment for a possible improvement.

as discussed on IRC, I'm not feeling very comfortable with the format stuff (in RawExtrinsicMetadataCore) but meh.

thx