We've run mypy on dependent modules and they seem to be happy enough.

The factory type annotations are in D8024.

I've scheduled the archival of the 7377 repos in one of the leftover one-shot queues.

So, as was mentioned during the irc discussion, one of the possible ways forward is to:

Thanks!

All production loaders have been restarted now.

After killing the minio data store, we've decided (as we had planned anyway) to start again from a clean slate.

I've restarted the staging workers (loader_git and loader_high_priority) with the new dulwich version

In T4311#86497, @ardumont wrote:

fwiw, jenkins is python3-dulwich aware.

I checked that the swh.loader.git tests are green with the new dulwich version.

We've asked for another bump at https://github.com/pypa/pypi-support/issues/1998.

Thanks for working on this, I think getting these insights will be useful.

4.a. properly declare this in the maven dependencies of swh.graph
4.b. ensure the container image generation pipeline and container entrypoint script properly handle this extra dependency and argument

Oops, thanks!

That's deployed on all nodes and validated as working now.

We had a mysterious error 500 issue with one buggy repository in the bitbucket API in the past, which we had reported to their Jira to no avail, but someone at Octobus (I think it was @marmoute or @Alphare ?) managed to reach out to one of the devs at atlassian to actually debug and fix the issue.

It may be worth adding a test for a loader getting a BaseException derivate in one of the early methods too. But that's a bit nitpicky.

(you probably want to update the commit message because you're not reverting the other commit anymore !)

Puppet ran manually on staging workers now, and the new statsd exporter has been deployed. The metrics properly show up in prometheus.

My debian package attempts went nowhere, it seems that one of the 3rd level dependencies needs a newer golang than what's available in buster-backports, so we'll get rid of the (homegrown anyway) debian package and go the upstream binaries route.

I expect that the clean celery worker termination command would also be useful for the current prod setup

(basically, the liveness probe could do what our current swh-worker-ping-restart script does: call swh scheduler celery-monitor --pattern "$celery_name" ping-workers a couple of times. We just need to figure out how to generate the $celery_name value consistently between the entrypoint and the liveness probe)

What component is providing the health http endpoint in the pod?

Yeah, that's definitely too verbose for normal operation!

Yeah, I'm not saying it should happen now, but the way this stuff has been handled has been inconsistent/painful/annoying me for a while, so maybe it's time?

I'm a bit uncomfortable with adding more ad-hoc logger.setLevel calls deep in SWH code (which then make reasoning about the logging config harder), rather than ripping it all apart and properly setting up the logging machinery "the python way" with a clean, explicit dictConfig (potentially pulled from a yaml file set as an envvar?).

*yeet*

Reset commit dates

Rebase; rename all raw_message fields to message_source

In D7799#202967, @vlorentz wrote:

what about "message_source" instead of "raw_message"? IMO it's clearer

Rebase and add docstring

🤷🤷🤷 let's try it and see...