In D6570#171059, @douardda wrote:

In D6570#170758, @vlorentz wrote:

You checked on the whole journal, right?

not yet. The shouldn't be any left, but... I'll check as soon as I have credentials for the production kafka cluster.

Bump the dependency on swh-journal to 0.9

In D6570#170941, @olasd wrote:

While we're here, could you check whether the other fixers are really still needed too?

In D6570#170758, @vlorentz wrote:

You checked on the whole journal, right?

^x^s...

typo

Overall it looks very good to me. There is room for improvement, for sure, but let's put this to work and see how it performs.

Document a bit more the value_deserializer and add a test for it

asking for review even if tests are expected to fail because it depends on D6565 (in swh-journal)

In D6564#170522, @ardumont wrote:

why if i may ask?

A bit sad, but hey, thanks!

Let's consider this is done since 2 of 3 bullets have been canceled (for now)

rebase

and fix the LOGGER.error usage

remove hardcoded log levels

Use ReadOnlyObjStorage and NonIterableObjStorage instead of NotImplementedError

document the build_objstorage() test helper function

remove useless statement

remove mistakenly commited mypy.ini file

rebase

remove a (comment) garbage line

rebase

slight simplification as suggested by vlorentz

respawn jenkins

respawn jenkins

rebase

split the diff in 2

B3 I am not convinced a "synthetic" flag on the Snapshot branch makes sense, or at least I find this name confusing, especially considering we already have a synthetic flag on Revision: it's not synthetic in the sense of it's not object crafted by SWH, it comes from the origin.

Ok I think what puzzle me in this description is the fact the 2 first bullets of the "git loader adaptations" are actually only one point: at the end of a successful loading, store a mapping in the extid table.

forgotten print statement...

In T3627#71819, @vlorentz wrote:

An alternative to annotating synthetic refs: add a "type" or "forge_type" attribute to snapshots.

What's the difference in deployed dependencies versions (staging vs. prod)?

For ENEA I'd llike to test different scenarios for the source objstorage:

just a quick remark about the scheduling of (sub)tasks of this task: IMHO the autoscaling should come last; all the supervision/monitoring/logging related tasks are much more important than the autoscaling.

Better docstrings and kill a few map()

Thx

be a bit more consistent...

In D6410#167121, @douardda wrote:

In D6410#166398, @ardumont wrote:

as @vlorentz pointed out [1], this change should be irrelevant though...

[1] https://github.com/celery/kombu/blob/master/kombu/serialization.py#L369-L372

does not seem to be the proper fix.

FTR, using the celery cli tool directly from a development venv to interact with the celery server running in the docker compose test setup (as described there ) used to work ok, but not any more.

One have to specify the app, like:

celery --app=swh.scheduler.celery_backend.config.app status

[edit] I use celery 5.1.2 in my venv.

Unless I'm mistaken, this error does not appear in sentry any more, right?

In D6410#166398, @ardumont wrote:

as @vlorentz pointed out [1], this change should be irrelevant though...

[1] https://github.com/celery/kombu/blob/master/kombu/serialization.py#L369-L372

looks fine to me

Ok but see my 2 (nitpicky) comments

LGTM thx

[FTR] See https://forge.softwareheritage.org/source/swh-perfecthash/