FTR, many of the cli.py (or cli/ packages) do import way too much stuff by default.
For this swh command to behave half decently, we need to be as lazy as possible regarding importing stuff, i.e. only import when needed, in @cli.command functions.

is this task still "a thing"? @olasd ?

do we mean something else than echo swh:1:rev:$(git rev-parse HEAD) ?
Do we want support for other (supported) VCS?

thx

FTR, I've run the launchpad lister in a docker and it executed fine, with fine being "it created 19340 load-git tasks"

closed by 8c937da20785699ae2a0a604104a9e458eced201

rebase

rebase + typos in ci msg

Add the magic touch

includes seirl's remarks (well the easy part of them)

In D3871#95682, @seirl wrote:

Thanks, this looks good!

Apart from minor nitpicks, my only concern here is that it might not be 100% clear who this doc is addressing. I reckon that people clicking on the "quickstart" page of swh.graph can come from different situations:

• people who want to use swh-graph without any consideration for the rest of the SWH infrastructure on an arbitrary graph (what you seem to have written here)
• people who want to install swh-graph as part of their SWH development environment
• people who would like to run topology experiments on our graph but have no idea where to start
• ...

In that regard, I am not sure "quickstart" is appropriate here -- but I really don't have a better idea of how this doc should be written. I am okay for merging that as-is until we figure out what we want to show potential users. Maybe @zack would like to chime in here?

I guess it would probably be a good idea to at least start with an intro paragraph about what we're trying to achieve in this tutorial?

okay-ish. I still don't really get the point of this diff. You still have as many fixtures as before, it's just that they all have the same same. For me it defies the simplification/clarification this diff claims it aims at. I won't reject but I'm not convinced neither (sorry).

looks ok but I find the commit message too cryptic.

I'm not sure I get what problem this diff is solving (and the commit message gives no clue about it).
The "why", focus on the "why" (and "how", if needed) :-)

we could argue that having the launchpad lister attempting a connection on instantiation is pretty bad, but yeah, ok

I guess this task can be closed, since this backfilling process will be part of the one we will run soon to fill the new kafka cluster

what's the status of this task today? is there a probe that tracks these broken links? (or a script one can run)

what's missing for this task to be closed?

I guess this also depends on a packagist loader, which we do not have at all for now...

rebase + anlambert's comment

In D3841#94881, @anlambert wrote:

While you are are on it, can you also remove the objstorage dependency in swh-web service definition inside docker-compose.yml ?

On the other hand, it will be computationally expensive, especially for contents.
What do you think?

Also beware that the default pagination value in the gitea lister is 3 (https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitea/lister.py$23) so it is very slow.

Ok I was expecting something a bit smart in explore.sapk.fr, but not really:

now we have the gitea lister, we should (upgrade swh.lister on prod and) add a few listing tasks, like this fsfe instance, as well as other instances like https://codeberg.org.

small improvements

remove the webapp->objstorage RPC link

In D3838#94866, @douardda wrote:

general-architecture.svg355 KBDownload

In D3838#94841, @anlambert wrote:

For the record, there is not really a RPC link between the webapp and the objstorage as its use is wrapped by the content_* methods of the storage interface.
So I think you should remove that new link.

improve the diagram

rebase

type annotate the new ensure_check_config helper function

rebase + add closes T2525 in the ci msg

rebase

rebase

rebase

rebase

rebase

pick the correct commit to be stashed...

rebase

rebase

fix failed black magic as ween by ardumont

rebase