Paths

Table of Contentst

Differential D7751

Add `algos.identifier.identifiers_missing`
Changes PlannedPublicDraft
Actions

Authored by Ericson2314 on May 5 2022, 8:13 PM.

Details

Reviewers

None

Group Reviewers

Reviewers

Summary

This is a small wrapper around the underlying *_missing functions in
order to assist with looking up missing SWHIDs.

This will be a building block to clean up some things in swh-web so that
https://forge.softwareheritage.org/T4224 doesn't come back.

Diff Detail

Repository

rDSTO Storage manager

Branch

identifiers-missing (branched from master)

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 29103
Build 45501: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 45500: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	527 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.algos.test_identifer::test_missing_contents[6\xfa\xdew\x19<\xb6\xd2\xbd\x82aa\xa0\x97\x9dd\xc2\x8a\xb4\xfa] swh_storage = <swh.storage.postgresql.storage.Storage object at 0x7f6c28288c50> content_id = b'6\xfa\xdew\x19<\xb6\xd2\xbd\x82aa\xa0\x97\x9dd\xc2\x8a\xb4\xfa' missing_id = b'\xb3\x93b\x1d\xd7\xb0\xad\x02MYy\x8c8\xcc\xbd\x19\xa5\xde\xb1_'
	522 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.algos.test_identifer::test_missing_contents[\xc92\xc7d\x9cm\xfaK\x822}\x12\x12\x15\x11i\t\xeb;\xea] swh_storage = <swh.storage.postgresql.storage.Storage object at 0x7f6c851c94e0> content_id = b'\xc92\xc7d\x9cm\xfaK\x822}\x12\x12\x15\x11i\t\xeb;\xea' missing_id = b'<\xa7Z\x95\xac\x0e!\x19\xca\x9a\x13\x90@o\x83j\xf0\x0eB\xd6'
	481 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.algos.test_identifer::test_missing_contents[\xd8\x1c\xc0q\x0e\xb6\xcf\x9e\xfd[\x92\n\x84S\xe1\xe0qW\xb6\xcd] swh_storage = <swh.storage.postgresql.storage.Storage object at 0x7f6c096d8198> content_id = b'\xd8\x1c\xc0q\x0e\xb6\xcf\x9e\xfd[\x92\n\x84S\xe1\xe0qW\xb6\xcd' missing_id = b"\x1f\x84\xd5'\x92\xb1 4\xfe\xe0\x1fBK\xa3\x88\xe3\xa3hD\xcc"
	414 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.algos.test_identifer::test_missing_directories[K\x82]\xc6B\xcbn\xb9\xa0`\xe5K\xf8\xd6\x92\x88\xfb\xeeI\x04] swh_storage = <swh.storage.postgresql.storage.Storage object at 0x7f6c07c7add8> directory_id = b'K\x82]\xc6B\xcbn\xb9\xa0`\xe5K\xf8\xd6\x92\x88\xfb\xeeI\x04' missing_id = b'\xb1\x04\x16\xc8\x87\x9d\x14\x0bK\xffRy\xf9I\x91\x96\xa7\xd8\xee\xab'
	344 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.algos.test_identifer::test_missing_directories[RV\xe8V\xa0\xa0\x89\x89f\xd6\xba\x14\xfe\xb48\x8b\x8b\x82\xd3\x02] swh_storage = <swh.storage.postgresql.storage.Storage object at 0x7f6c3c3330f0> directory_id = b'RV\xe8V\xa0\xa0\x89\x89f\xd6\xba\x14\xfe\xb48\x8b\x8b\x82\xd3\x02' missing_id = b'Bd;\xd1&h\xc0Hyg\xc4\xd1\xad\xf4\x9f\x18\x98\x11\xec\xf1'
		View Full Test Results (23 Failed · 1,219 Passed · 40 Skipped)

Event Timeline

Ericson2314 created this revision.May 5 2022, 8:13 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptMay 5 2022, 8:13 PM

Fix spelling error

Build has FAILED

Patch application report for D7751 (id=28025)

Rebasing onto 9562953493...

Current branch diff-target is up to date.

Changes applied before test

commit 31d4c3e4cfee780a1e3c823ef132e216705d8704
Author: John Ericson <John.Ericson@Obsidian.Systems>
Date:   Thu May 5 13:25:57 2022 -0400

    Add `algos.identifier.identifiers_missing`
    
    This is a small wrapper around the underlying `*_missing` functions in
    order to assist with looking up missing SWHIDs.
    
    This will be a building block to clean up some things in swh-web so that
    https://forge.softwareheritage.org/T4224 doesn't come back.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1605/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1605/console

Harbormaster failed remote builds in B29101: Diff 28025!May 5 2022, 8:15 PM

Build has FAILED

Patch application report for D7751 (id=28026)

Rebasing onto 9562953493...

Current branch diff-target is up to date.

Changes applied before test

commit 026af46d2e36b5c40309ad1a1c13549167c2ff2e
Author: John Ericson <John.Ericson@Obsidian.Systems>
Date:   Thu May 5 13:25:57 2022 -0400

    Add `algos.identifier.identifiers_missing`
    
    Summary:
    This is a small wrapper around the underlying `*_missing` functions in
    order to assist with looking up missing SWHIDs.
    
    This will be a building block to clean up some things in swh-web so that
    https://forge.softwareheritage.org/T4224 doesn't come back.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D7751

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1606/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1606/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 5 2022, 8:16 PM

Harbormaster failed remote builds in B29102: Diff 28026!

Ensure thre is an error on invalid object type

Condense / deduplicate to the code too in the process.

Build has FAILED

Patch application report for D7751 (id=28027)

Rebasing onto 9562953493...

Current branch diff-target is up to date.

Changes applied before test

commit a92cef818b5a2b947f2cf733283c3f28116a2797
Author: John Ericson <John.Ericson@Obsidian.Systems>
Date:   Thu May 5 13:25:57 2022 -0400

    Add `algos.identifier.identifiers_missing`
    
    Summary:
    This is a small wrapper around the underlying `*_missing` functions in
    order to assist with looking up missing SWHIDs.
    
    This will be a building block to clean up some things in swh-web so that
    https://forge.softwareheritage.org/T4224 doesn't come back.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D7751

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1607/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1607/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 5 2022, 8:30 PM

Harbormaster failed remote builds in B29103: Diff 28027!

Need to drain iterable to set

Build was aborted

Patch application report for D7751 (id=28028)

Rebasing onto 9562953493...

Current branch diff-target is up to date.

Changes applied before test

commit 1717c3eaa70a5c8e4c73dd6ac59926e8dd9f3f8e
Author: John Ericson <John.Ericson@Obsidian.Systems>
Date:   Thu May 5 13:25:57 2022 -0400

    Add `algos.identifier.identifiers_missing`
    
    Summary:
    This is a small wrapper around the underlying `*_missing` functions in
    order to assist with looking up missing SWHIDs.
    
    This will be a building block to clean up some things in swh-web so that
    https://forge.softwareheritage.org/T4224 doesn't come back.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D7751

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1608/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1608/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 5 2022, 8:54 PM

Harbormaster failed remote builds in B29104: Diff 28028!

Ericson2314 mentioned this in D7749: Cleanup `archive.lookup_missing_hashes` and `api_swhid_known`.May 6 2022, 5:17 PM