remove the pprint from the doctest

In D5589#142405, @ardumont wrote:

if you are convinced it's a report covering issue, ok then ;)

I would not say all this is crystal clear to me, but overall looks fine.

lgtm

In D5614#142400, @vlorentz wrote:

Why only 0o100644 and 0o100755? I don't think we should make the package loaders discard this information just because Git does.

Rebased and updated to current HEAD

rebase

also add lik to the docker page

it works! thx

actually no, it's not the same...

Examples of such missing objects are revisions with attributes that cannot fit the current data model, e.g. out of range dates. We have example of such revisions in kafka, as mentionned in T3200 and T3170.

Same as T1957

These revisions are probably coming from https://gitlab.com/gitlab-org/gitlab-test (or a clone)

Ah fun, one of the revisions with this pb, on staging (ba3343bc4fa403a8dfbfcab7fc1a8c29ee34bd69) seems to have been crafted by https://gitlab.com/gitlab-org/gitlab-foss/-/blob/staging-26-fix_add_deploy_key_spec/spec/models/merge_request_diff_commit_spec.rb

Note that none of their parent revisions can be found either in the archive (one invalid revision in a set of ingested revisions prevent any of them being inserted in the database I suppose, but they are already inserted in kafka at this moment).

See T3170 (error generated by the same invalid kafka messages).

So what about exports of the archive available on git-annex?

In T3246#63559, @douardda wrote:

do we also intent to have a takedown topic on kafka?

Note that there is the same transient vs cumulative discrepency on the "Accepted requests" graph.

In T1481#63785, @douardda wrote:

I think the "submitted requests per visit type / status" graph should be split in 2 parts. Both accepted and rejected are cumulative values that will indefinitely grow, while pending are transient value aiming at staying near zero, so it makes no sense to have them on the same graph.

I think the "submitted requests per visit type / status" graph should be split in 2 parts. Both accepted and rejected are cumulative values that will indefinitely grow, while pending are transient value aiming at staying near zero, so it makes no sense to have them on the same graph.

In T3084#63569, @douardda wrote:

is there a grafana dashboard dedicated to this queue?

In D5553#140866, @anlambert wrote:

Looks good to me. It's true the black version we are using is quite outdated.

We should upgrade black in all other swh repositories in the same manner then.

rebas

rebase

rebased

rebased

The loading tasks created during this first listing were oneshot tasks. So they have been modified to recurring tasks with something like:

Doesn't this deserve a state-of-the-art kind of thing? Are there documentation material on the subject? How does other (big) cassandra users handle this?

The listing task has been disabled, I think because of failures in the last executions:

is there a grafana dashboard dedicated to this queue?

also: what about exports we provide on git annex?

do we also intent to have a takedown topic on kafka?

Just got this one below. Note that this occurred just when the replayer actually started to insert object in the storage (before that, since the start of the replayer process, only kafka scaffolding took place for quite some time, around 30mn!)

looks like there is no revision with date or committer_date > 9999-12-31 in the main storage...

fix commit message

In D5413#137649, @douardda wrote:

In D5413#137363, @vlorentz wrote:

Could you add a test for the storage? All other *_add have a journal test IIRC

let me check that