rebase

rebase + typos in ci msg

Add the magic touch

includes seirl's remarks (well the easy part of them)

In D3871#95682, @seirl wrote:

Thanks, this looks good!

Apart from minor nitpicks, my only concern here is that it might not be 100% clear who this doc is addressing. I reckon that people clicking on the "quickstart" page of swh.graph can come from different situations:

• people who want to use swh-graph without any consideration for the rest of the SWH infrastructure on an arbitrary graph (what you seem to have written here)
• people who want to install swh-graph as part of their SWH development environment
• people who would like to run topology experiments on our graph but have no idea where to start
• ...

In that regard, I am not sure "quickstart" is appropriate here -- but I really don't have a better idea of how this doc should be written. I am okay for merging that as-is until we figure out what we want to show potential users. Maybe @zack would like to chime in here?

I guess it would probably be a good idea to at least start with an intro paragraph about what we're trying to achieve in this tutorial?

okay-ish. I still don't really get the point of this diff. You still have as many fixtures as before, it's just that they all have the same same. For me it defies the simplification/clarification this diff claims it aims at. I won't reject but I'm not convinced neither (sorry).

looks ok but I find the commit message too cryptic.

I'm not sure I get what problem this diff is solving (and the commit message gives no clue about it).
The "why", focus on the "why" (and "how", if needed) :-)

we could argue that having the launchpad lister attempting a connection on instantiation is pretty bad, but yeah, ok

I guess this task can be closed, since this backfilling process will be part of the one we will run soon to fill the new kafka cluster

what's the status of this task today? is there a probe that tracks these broken links? (or a script one can run)

what's missing for this task to be closed?

I guess this also depends on a packagist loader, which we do not have at all for now...

rebase + anlambert's comment

In D3841#94881, @anlambert wrote:

While you are are on it, can you also remove the objstorage dependency in swh-web service definition inside docker-compose.yml ?

On the other hand, it will be computationally expensive, especially for contents.
What do you think?

Also beware that the default pagination value in the gitea lister is 3 (https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitea/lister.py$23) so it is very slow.

Ok I was expecting something a bit smart in explore.sapk.fr, but not really:

now we have the gitea lister, we should (upgrade swh.lister on prod and) add a few listing tasks, like this fsfe instance, as well as other instances like https://codeberg.org.

small improvements

remove the webapp->objstorage RPC link

In D3838#94866, @douardda wrote:

general-architecture.svg355 KBDownload

In D3838#94841, @anlambert wrote:

For the record, there is not really a RPC link between the webapp and the objstorage as its use is wrapped by the content_* methods of the storage interface.
So I think you should remove that new link.

improve the diagram

rebase

type annotate the new ensure_check_config helper function

rebase + add closes T2525 in the ci msg

rebase

rebase

rebase

rebase

rebase

pick the correct commit to be stashed...

rebase

rebase

fix failed black magic as ween by ardumont

rebase

rebase

ok but please explain why in your commit message...

as reported by vlorentz, duplicate of D3811

as said in the commit message, It must be updated jointly in 30-swh-schema.sql
(including after a rebase...)

Just to make is clear, my main objective here is to have a seatbelt for the mirror scenario: prevent a mirror with updated code but not-yet-updated database from even running until the DB migration script has been executed.