Page MenuHomeSoftware Heritage

Stop storing authority/fetcher metadata.
ClosedPublic

Authored by vlorentz on May 4 2021, 4:06 PM.

Details

Summary

We still don't have a use for them, and they are causing issues;
such as being unable to add an authority/fetcher based only
on a REMD object, which is needed by the replayer.

Resolves T3303.

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build has FAILED

Patch application report for D5670 (id=20256)

Rebasing onto 051b771523...

Current branch diff-target is up to date.
Changes applied before test
commit e2afe93b756bd073ff855bfc45d0b1baa197f22e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1301/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1301/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 4 2021, 4:16 PM
Harbormaster failed remote builds in B21243: Diff 20256!

maybe add a "fixer" also (in fixers.py) for the replayer to not break because of a metadata field in a kafka message?

@douardda It won't break, because the metadata field will simply be ignored.

But I'll add a fixer in a future diff, so I can remove the field from the model.

Build was aborted

Patch application report for D5670 (id=20263)

Rebasing onto 051b771523...

Current branch diff-target is up to date.
Changes applied before test
commit f9ca2fa32a122c8e839b7ef252a6b2813b971328
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1303/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1303/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 4 2021, 5:25 PM
Harbormaster failed remote builds in B21250: Diff 20263!

Build is green

Patch application report for D5670 (id=20263)

Rebasing onto 051b771523...

Current branch diff-target is up to date.
Changes applied before test
commit f9ca2fa32a122c8e839b7ef252a6b2813b971328
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1304/ for more details.

But I'll add a fixer in a future diff, so I can remove the field from the model.

Actually, that's not even needed because that field was not written to Kafka.

This revision is now accepted and ready to land.May 5 2021, 12:34 PM

Build is green

Patch application report for D5670 (id=20291)

Rebasing onto 77ef651d95...

Current branch diff-target is up to date.
Changes applied before test
commit e170fb29fde7ddc896b28f60505f64adf3631410
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue May 4 16:04:38 2021 +0200

    Stop storing authority/fetcher metadata.
    
    We still don't have a use for them, and they are causing issues;
    such as being unable to add an authority/fetcher based only
    on a REMD object, which is needed by the replayer.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1308/ for more details.

This revision was automatically updated to reflect the committed changes.