Add more origins to savecodenow::e2e.
ClosedPublic
Actions

Authored by vlorentz on Sep 27 2022, 11:33 AM.

Details

Reviewers

ardumont
olasd

Group Reviewers

System administrators

Maniphest Tasks

Restricted Maniphest Task

Commits

rSPSITE7f082ccb7edf: Add more origins to savecodenow::e2e.

Summary

This will reduce the number of visits on the tested origins,
by randomly selecting one to submit to SCN, instead of always
visiting the same one.

Because of the current performance of swh-storage, having too
many visits for the same origin causes timeouts.

Depends on D8544.

Fixes T4386#91104

Diff Detail

Repository

rSPSITE puppet-swh-site

Branch

production

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 31808
Build 49777: arc lint + arc unit

Event Timeline

vlorentz created this revision.Sep 27 2022, 11:33 AM

Herald added a reviewer: System administrators. · View Herald TranscriptSep 27 2022, 11:33 AM

Harbormaster completed remote builds in B31786: Diff 30810.Sep 27 2022, 11:33 AM

vlorentz requested review of this revision.Sep 27 2022, 11:33 AM

One limitation i see is an eventual flip-flop about the ingestion time (which was, afair, why we did not implement the origin_random_get [in the last 2 months or so], in the storage endpoint in the first place).
That flip-flop could create false negatives in the alerting.

I don't have anything better to propose so fine.

@olasd, @vsellier anything to add?

data/common/common.yaml
2044	maybe remove the old ones (parmap, panda and hgview)

olasd accepted this revision.Sep 27 2022, 2:28 PM

olasd added inline comments.

data/common/common.yaml
2062	this one is inconsistent with the others

This revision is now accepted and ready to land.Sep 27 2022, 2:28 PM

vlorentz added inline comments.Sep 27 2022, 2:37 PM

data/common/common.yaml
2062	I used the URLs we already have in the archive

In D8546#221913, @ardumont wrote:

One limitation i see is an eventual flip-flop about the ingestion time (which was, afair, why we did not implement the origin_random_get [in the last 2 months or so], in the storage endpoint in the first place).
That flip-flop could create false negatives in the alerting.

That's why I hand-picked origins that should be stable.

olasd added inline comments.Sep 27 2022, 2:40 PM