Page MenuHomeSoftware Heritage

Add more origins to savecodenow::e2e.
ClosedPublic

Authored by vlorentz on Sep 27 2022, 11:33 AM.

Details

Reviewers
ardumont
olasd
Group Reviewers
System administrators
Maniphest Tasks
Restricted Maniphest Task
Commits
rSPSITE7f082ccb7edf: Add more origins to savecodenow::e2e.
Summary

This will reduce the number of visits on the tested origins,
by randomly selecting one to submit to SCN, instead of always
visiting the same one.

Because of the current performance of swh-storage, having too
many visits for the same origin causes timeouts.

Depends on D8544.

Fixes T4386#91104

Diff Detail

Repository
rSPSITE puppet-swh-site
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont added subscribers: vsellier, olasd, ardumont.

One limitation i see is an eventual flip-flop about the ingestion time (which was, afair, why we did not implement the origin_random_get [in the last 2 months or so], in the storage endpoint in the first place).
That flip-flop could create false negatives in the alerting.

I don't have anything better to propose so fine.

@olasd, @vsellier anything to add?

data/common/common.yaml
2052

maybe remove the old ones (parmap, panda and hgview)

olasd added inline comments.
data/common/common.yaml
2070

this one is inconsistent with the others

This revision is now accepted and ready to land.Sep 27 2022, 2:28 PM
data/common/common.yaml
2070

I used the URLs we already have in the archive

One limitation i see is an eventual flip-flop about the ingestion time (which was, afair, why we did not implement the origin_random_get [in the last 2 months or so], in the storage endpoint in the first place).
That flip-flop could create false negatives in the alerting.

That's why I hand-picked origins that should be stable.

data/common/common.yaml
2070

ok. Please add a comment so that we don't "fix" it later?

This revision was automatically updated to reflect the committed changes.