Page MenuHomeSoftware Heritage

Stop storing authority/fetcher metadata.
ClosedPublic

Authored by vlorentz on May 4 2021, 4:06 PM.

Details

Summary

We still don't have a use for them, and they are causing issues;
such as being unable to add an authority/fetcher based only
on a REMD object, which is needed by the replayer.

Resolves T3303.

Diff Detail

Repository
rDSTO Storage manager
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 21277
Build 33041: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 33040: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D5670 (id=20256)

Rebasing onto 051b771523...

Current branch diff-target is up to date.
Changes applied before test
commit e2afe93b756bd073ff855bfc45d0b1baa197f22e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1301/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1301/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 4 2021, 4:16 PM
Harbormaster failed remote builds in B21243: Diff 20256!

maybe add a "fixer" also (in fixers.py) for the replayer to not break because of a metadata field in a kafka message?

@douardda It won't break, because the metadata field will simply be ignored.

But I'll add a fixer in a future diff, so I can remove the field from the model.

Build was aborted

Patch application report for D5670 (id=20263)

Rebasing onto 051b771523...

Current branch diff-target is up to date.
Changes applied before test
commit f9ca2fa32a122c8e839b7ef252a6b2813b971328
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1303/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1303/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 4 2021, 5:25 PM
Harbormaster failed remote builds in B21250: Diff 20263!

Build is green

Patch application report for D5670 (id=20263)

Rebasing onto 051b771523...

Current branch diff-target is up to date.
Changes applied before test
commit f9ca2fa32a122c8e839b7ef252a6b2813b971328
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1304/ for more details.

But I'll add a fixer in a future diff, so I can remove the field from the model.

Actually, that's not even needed because that field was not written to Kafka.

This revision is now accepted and ready to land.May 5 2021, 12:34 PM

Build is green

Patch application report for D5670 (id=20291)

Rebasing onto 77ef651d95...

Current branch diff-target is up to date.
Changes applied before test
commit e170fb29fde7ddc896b28f60505f64adf3631410
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1308/ for more details.

This revision was automatically updated to reflect the committed changes.