Page MenuHomeSoftware Heritage

identifiers: Expose manifest computation
ClosedPublic

Authored by vlorentz on Apr 30 2021, 11:30 AM.

Details

Summary

Before this commit, manifests were only computed internally before
hashing, so they were not available to outside modules.

This makes testing the module very painful, because identifier
functions can only be tested by checking the hash; so test failures
did not show mismatches between the computed manifest and
the expected one.

Additionally, the 'git bare cooker' of the vault is likely to use
these as well, as it needs to format git objects in the same format.

Test Plan

Should I add specific tests for the new manifest functions?
They are indirectly tested by the existing tests.

Diff Detail

Repository
rDMOD Data model
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 21171
Build 32859: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 32858: arc lint + arc unit

Unit TestsFailed

TimeTest
1 msJenkins > .tox.py3-full.lib.python3.7.site-packages.swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier::test_epoch
self = <swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier testMethod=test_epoch> def test_epoch(self):
1 msJenkins > .tox.py3-full.lib.python3.7.site-packages.swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier::test_maximal
self = <swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier testMethod=test_maximal> def test_maximal(self):
1 msJenkins > .tox.py3-full.lib.python3.7.site-packages.swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier::test_minimal
self = <swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier testMethod=test_minimal> def test_minimal(self):
1 msJenkins > .tox.py3-full.lib.python3.7.site-packages.swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier::test_negative_epoch
self = <swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier testMethod=test_negative_epoch> def test_negative_epoch(self):
1 msJenkins > .tox.py3-full.lib.python3.7.site-packages.swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier::test_negative_timestamp
self = <swh.model.tests.test_identifiers.RawExtrinsicMetadataIdentifier testMethod=test_negative_timestamp> def test_negative_timestamp(self):
View Full Test Results (6 Failed · 464 Passed · 2 Skipped)

Event Timeline

Build has FAILED

Patch application report for D5650 (id=20181)

Rebasing onto f7e9d5c290...

Current branch diff-target is up to date.
Changes applied before test
commit 4eb74157df01fddb980850671b33cae51a2cdb44
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 30 11:26:32 2021 +0200

    identifiers: Expose manifest computation
    
    Before this commit, manifests were only computed internally before
    hashing, so they were not available to outside modules.
    
    This makes testing the module very painful, because identifier
    functions can only be tested by checking the hash; so test failures
    did not show mismatches between the computed manifest and
    the expected one.
    
    Additionally, the 'git bare cooker' of the vault is likely to use
    these as well, as it needs to format git objects in the same format.

Link to build: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/324/
See console output for more information: https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/324/console

Harbormaster returned this revision to the author for changes because remote builds failed.Apr 30 2021, 11:31 AM
Harbormaster failed remote builds in B21171: Diff 20181!

Build is green

Patch application report for D5650 (id=20185)

Rebasing onto f7e9d5c290...

Current branch diff-target is up to date.
Changes applied before test
commit 3a4f484feaccd569a0b9dc37531d7af50bc3871e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 30 11:26:32 2021 +0200

    identifiers: Expose manifest computation
    
    Before this commit, manifests were only computed internally before
    hashing, so they were not available to outside modules.
    
    This makes testing the module very painful, because identifier
    functions can only be tested by checking the hash; so test failures
    did not show mismatches between the computed manifest and
    the expected one.
    
    Additionally, the 'git bare cooker' of the vault is likely to use
    these as well, as it needs to format git objects in the same format.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/325/ for more details.

olasd added a subscriber: olasd.

D5652 adds tests for the manifest format, so I don't think you need to do that here.

This revision is now accepted and ready to land.Apr 30 2021, 5:39 PM

Build is green

Patch application report for D5650 (id=20203)

Rebasing onto df036ef1c3...

Current branch diff-target is up to date.
Changes applied before test
commit 8221147ce6e74b3f685baead1906675b3454065c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 30 11:26:32 2021 +0200

    identifiers: Expose manifest computation
    
    Before this commit, manifests were only computed internally before
    hashing, so they were not available to outside modules.
    
    This makes testing the module very painful, because identifier
    functions can only be tested by checking the hash; so test failures
    did not show mismatches between the computed manifest and
    the expected one.
    
    Additionally, the 'git bare cooker' of the vault is likely to use
    these as well, as it needs to format git objects in the same format.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/331/ for more details.

Build is green

Patch application report for D5650 (id=20475)

Rebasing onto 31cb72e9f0...

Current branch diff-target is up to date.
Changes applied before test
commit 523ab642bb48491f8e09fb61cc32a9e11420b556
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 30 11:26:32 2021 +0200

    identifiers: Expose manifest computation
    
    Before this commit, manifests were only computed internally before
    hashing, so they were not available to outside modules.
    
    This makes testing the module very painful, because identifier
    functions can only be tested by checking the hash; so test failures
    did not show mismatches between the computed manifest and
    the expected one.
    
    Additionally, the 'git bare cooker' of the vault is likely to use
    these as well, as it needs to format git objects in the same format.

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/334/ for more details.