Page MenuHomeSoftware Heritage

storage*: Rename and type content_get_data(Sha1) -> Optional[bytes]
ClosedPublic

Authored by ardumont on Aug 6 2020, 10:58 PM.

Details

Summary

Rename the confusing endpoint content_get to content_get_data. This also
now works on one content as it is how it is used today.

Impacts:

grep -r "content_get(" $SWH_ENVIRONMENT_HOME/**/swh/* | grep -v swh-storage
/home/tony/work/inria/repo/swh/swh-environment/swh-indexer/swh/indexer/tests/utils.py:    def test_generate_content_get(self):
/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/swh/loader/package/npm/tests/test_npm.py:        list(loader.storage.content_get(_expected_new_contents_first_visit))
/home/tony/work/inria/repo/swh/swh-environment/swh-vault/swh/vault/to_disk.py:    contents_fetched = storage.content_get(contents_to_fetch)
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/common/service.py:            c = _first_element(storage.content_get([content.sha1]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/common/service.py:    content = _first_element(storage.content_get([content_sha1_bytes]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/browse/views/test_content.py:    content_data = archive_data.content_get(content["sha1"])["data"]
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/browse/views/test_content.py:    content_data = archive_data.content_get(content["sha1"])["data"]
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/browse/views/test_content.py:    content_data = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/api/views/test_content.py:    expected_data = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/api/views/test_content.py:    expected_data = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/data.py:        cnt = next(storage.content_get([sha1]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/common/test_service.py:    expected_data = archive_data.content_get(expected_dir_entry["checksums"]["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/common/test_service.py:    expected_content = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/conftest.py:    def content_get(self, cnt_id):
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/conftest.py:        cnt = next(self.storage.content_get([cnt_id_bytes]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/strategies.py:            get_tests_data()["storage"].content_get([hash_to_bytes(c["sha1"])])
Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

swh/storage/tests/test_retry.py
50

It doesn't get more confusing than that.
Here the author most probably meant content_get_metadata and not this one.

I won't change it now though.
Later when i'm done renaming those endpoint appropriately.

Build is green

Patch application report for D3728 (id=13132)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit 72c5af322e6dcccace9b92cb1bf8fb4fbf98c306
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/707/ for more details.

ardumont edited the summary of this revision. (Show Details)

Build is green

Patch application report for D3728 (id=13138)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit 8dc21c9b41903a598c08bcbcb114f7ac2e9b1489
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/708/ for more details.

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/storage/cassandra/storage.py
169

keep the FIXME

swh/storage/in_memory.py
260

same

swh/storage/interface.py
33

move this to model

swh/storage/storage.py
272

same

This revision is now accepted and ready to land.Aug 7 2020, 9:22 AM
swh/storage/interface.py
33

yes, D3738

now model 0.6.6 has it.

ardumont edited the summary of this revision. (Show Details)

Adapt according to review (+ bump model)

Build has FAILED

Patch application report for D3728 (id=13156)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit f057deddfdf16101ffbfaac1ee7725baf288d911
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/711/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/711/console

Build has FAILED

Patch application report for D3728 (id=13156)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit f057deddfdf16101ffbfaac1ee7725baf288d911
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/713/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/713/console

Build is green

Patch application report for D3728 (id=13158)

Rebasing onto 243e873175...

Current branch diff-target is up to date.
Changes applied before test
commit bfa8f46ea44d05d5dce15071d164b368da3f975f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/714/ for more details.