Paths

Table of Contentst

Differential D7720

Add `algos.directory.directory_get_all_entries`
AbandonedPublic
Actions

Authored by Ericson2314 on May 1 2022, 8:02 PM.

Details

Reviewers

None

Group Reviewers

Reviewers

Summary

There isn't yet a separate test_directories because the infrastructure
in swh.model.hypothesis.strategies doesn't not exist yet. But there
was an existing test where we could use this.

Diff Detail

Repository

rDSTO Storage manager

Branch

directory-algo

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 28987
Build 45315: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 45314: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	2,249 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_api_client.TestStorageApi::test_directory_add_get_arbitrary self = <swh.storage.tests.test_api_client.TestStorageApi object at 0x7f4c7b3649e8> swh_storage = <RemoteStorage url=mock://example.com/>
	1,185 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_cassandra.TestCassandraStorage::test_directory_add_get_arbitrary self = <swh.storage.tests.test_cassandra.TestCassandraStorage object at 0x7f4c7a36f978> swh_storage = <swh.storage.cassandra.storage.CassandraStorage object at 0x7f4c5014c940>
	739 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_in_memory.TestInMemoryStorage::test_directory_add_get_arbitrary self = <swh.storage.tests.test_in_memory.TestInMemoryStorage object at 0x7f4c7a56a518> swh_storage = <swh.storage.in_memory.InMemoryStorage object at 0x7f4c7a56af60>
	924 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_postgresql.TestStorage::test_directory_add_get_arbitrary self = <swh.storage.tests.test_postgresql.TestStorage object at 0x7f4c7ac9f6a0> swh_storage = <swh.storage.postgresql.storage.Storage object at 0x7f4a27fe7d68>
	1,302 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_tenacious.TestTenaciousStorage::test_directory_add_get_arbitrary self = <swh.storage.tests.test_tenacious.TestTenaciousStorage object at 0x7f4c7a73f5f8> swh_storage = <swh.storage.proxies.tenacious.TenaciousProxyStorage object at 0x7f4a256f1f28>
		View Full Test Results (5 Failed · 1,207 Passed · 40 Skipped)

Event Timeline

Ericson2314 created this revision.May 1 2022, 8:02 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptMay 1 2022, 8:02 PM

Fix the copyright year

Ericson2314 mentioned this in D7192: Route for fetching Git-encoded objects.May 1 2022, 8:08 PM

Build has FAILED

Patch application report for D7720 (id=27923)

Rebasing onto fb5514112d...

Current branch diff-target is up to date.

Changes applied before test

commit 1ae734d0d9d0501d584728e19fed6bc3e3228921
Author: John Ericson <John.Ericson@Obsidian.Systems>
Date:   Sun May 1 13:48:10 2022 -0400

    Add `algos.directory.directory_get_all_entries`
    
    There isn't yet a separate `test_directories` because the infrastructure
    in `swh.model.hypothesis.strategies` doesn't not exist yet. But there
    was an existing test where we could use this.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1600/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1600/console

Harbormaster failed remote builds in B28987: Diff 27923!May 1 2022, 8:12 PM

Build has FAILED

Patch application report for D7720 (id=27924)

Rebasing onto fb5514112d...

Current branch diff-target is up to date.

Changes applied before test

commit 95c7b5f1f7d5a26cc70282663827a0e6b8805492
Author: John Ericson <John.Ericson@Obsidian.Systems>
Date:   Sun May 1 13:48:10 2022 -0400

    Add `algos.directory.directory_get_all_entries`
    
    Summary:
    There isn't yet a separate `test_directories` because the infrastructure
    in `swh.model.hypothesis.strategies` doesn't not exist yet. But there
    was an existing test where we could use this.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D7720

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1601/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1601/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 1 2022, 8:16 PM

Harbormaster failed remote builds in B28988: Diff 27924!

Fix test

Build is green

Patch application report for D7720 (id=27925)

Rebasing onto fb5514112d...

Current branch diff-target is up to date.

Changes applied before test

commit 10ff8e430fed82cc7583273dd1e6f49be8febc82
Author: John Ericson <John.Ericson@Obsidian.Systems>
Date:   Sun May 1 13:48:10 2022 -0400

    Add `algos.directory.directory_get_all_entries`
    
    Summary:
    There isn't yet a separate `test_directories` because the infrastructure
    in `swh.model.hypothesis.strategies` doesn't not exist yet. But there
    was an existing test where we could use this.
    
    Reviewers: #reviewers
    
    Differential Revision: https://forge.softwareheritage.org/D7720

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1602/ for more details.

Harbormaster completed remote builds in B28989: Diff 27925.May 1 2022, 9:09 PM

Ericson2314 requested review of this revision.May 1 2022, 9:09 PM

Good idea.

Regarding tests, avoid using hypothesis for new tests outside swh-web, it's slow and not reliable.
Instead, you should make tests insert objects directly in the storage

Do you think it would be possible to land this, and a version of the main one that depends on this, and then circle back to add the additional tests for malformed objects?

That would unblock the remaining work on the IPFS side so we can do more in parallel.

I opened this one right away because I figured it is more of a pain of move around code than just get what goes where right the first time, but additional tests I do think are easy enough to add incrementally.

vlorentz mentioned this in D7722: Add function storage.algos.directory.directory_get.May 2 2022, 12:26 PM

I did this in D7722 to spare a couple of loops of cross-TZ interaction

Ericson2314 abandoned this revision.May 2 2022, 4:29 PM