Page MenuHomeSoftware Heritage
Differential D3728

storage*: Rename and type content_get_data(Sha1) -> Optional[bytes]
ClosedPublic
Actions

Authored by ardumont on Aug 6 2020, 10:58 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDSTObfa8f46ea44d: storage*: Rename content_get_data(Sha1) -> Optional[bytes]
Summary

Rename the confusing endpoint content_get to content_get_data. This also
now works on one content as it is how it is used today.

Impacts:

grep -r "content_get(" $SWH_ENVIRONMENT_HOME/**/swh/* | grep -v swh-storage
/home/tony/work/inria/repo/swh/swh-environment/swh-indexer/swh/indexer/tests/utils.py:    def test_generate_content_get(self):
/home/tony/work/inria/repo/swh/swh-environment/swh-loader-core/swh/loader/package/npm/tests/test_npm.py:        list(loader.storage.content_get(_expected_new_contents_first_visit))
/home/tony/work/inria/repo/swh/swh-environment/swh-vault/swh/vault/to_disk.py:    contents_fetched = storage.content_get(contents_to_fetch)
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/common/service.py:            c = _first_element(storage.content_get([content.sha1]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/common/service.py:    content = _first_element(storage.content_get([content_sha1_bytes]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/browse/views/test_content.py:    content_data = archive_data.content_get(content["sha1"])["data"]
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/browse/views/test_content.py:    content_data = archive_data.content_get(content["sha1"])["data"]
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/browse/views/test_content.py:    content_data = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/api/views/test_content.py:    expected_data = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/api/views/test_content.py:    expected_data = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/data.py:        cnt = next(storage.content_get([sha1]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/common/test_service.py:    expected_data = archive_data.content_get(expected_dir_entry["checksums"]["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/common/test_service.py:    expected_content = archive_data.content_get(content["sha1"])
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/conftest.py:    def content_get(self, cnt_id):
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/conftest.py:        cnt = next(self.storage.content_get([cnt_id_bytes]))
/home/tony/work/inria/repo/swh/swh-environment/swh-web/swh/web/tests/strategies.py:            get_tests_data()["storage"].content_get([hash_to_bytes(c["sha1"])])
Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
fix-content-get
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 14425
Build 22180: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 22179: arc lint + arc unit

Event Timeline

ardumont created this revision.Aug 6 2020, 10:58 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptAug 6 2020, 10:58 PM
ardumont added inline comments.Aug 6 2020, 11:00 PM
swh/storage/tests/test_retry.py
50

It doesn't get more confusing than that.
Here the author most probably meant content_get_metadata and not this one.

I won't change it now though.
Later when i'm done renaming those endpoint appropriately.

swh-public-ci added a comment.Aug 6 2020, 11:03 PM

Build is green

Patch application report for D3728 (id=13132)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit 72c5af322e6dcccace9b92cb1bf8fb4fbf98c306
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/707/ for more details.

Harbormaster completed remote builds in B14398: Diff 13132.Aug 6 2020, 11:03 PM
ardumont mentioned this in D3729: npm: Fix assertion to use the correct storage api.Aug 6 2020, 11:13 PM
ardumont updated this revision to Diff 13138.Aug 7 2020, 12:42 AM

Rework commit message

ardumont edited the summary of this revision. (Show Details)Aug 7 2020, 12:44 AM
ardumont edited the summary of this revision. (Show Details)
swh-public-ci added a comment.Aug 7 2020, 12:46 AM

Build is green

Patch application report for D3728 (id=13138)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit 8dc21c9b41903a598c08bcbcb114f7ac2e9b1489
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/708/ for more details.

Harbormaster completed remote builds in B14404: Diff 13138.Aug 7 2020, 12:46 AM
vlorentz accepted this revision.Aug 7 2020, 9:22 AM
vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/storage/cassandra/storage.py
169

keep the FIXME

swh/storage/in_memory.py
260

same

swh/storage/interface.py
33

move this to model

swh/storage/storage.py
272

same

This revision is now accepted and ready to land.Aug 7 2020, 9:22 AM
ardumont added inline comments.Aug 7 2020, 10:56 AM
swh/storage/interface.py
33

yes, D3738

now model 0.6.6 has it.

ardumont updated this revision to Diff 13156.Aug 7 2020, 11:06 AM
ardumont edited the summary of this revision. (Show Details)

Adapt according to review (+ bump model)

swh-public-ci added a comment.Aug 7 2020, 11:11 AM

Build has FAILED

Patch application report for D3728 (id=13156)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit f057deddfdf16101ffbfaac1ee7725baf288d911
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/711/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/711/console

Harbormaster failed remote builds in B14421: Diff 13156!Aug 7 2020, 11:11 AM
swh-public-ci added a comment.Aug 7 2020, 11:38 AM

Build has FAILED

Patch application report for D3728 (id=13156)

Rebasing onto 653b1f9ada...

Current branch diff-target is up to date.
Changes applied before test
commit f057deddfdf16101ffbfaac1ee7725baf288d911
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/713/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/713/console

ardumont updated this revision to Diff 13158.Aug 7 2020, 12:27 PM

Rebase

swh-public-ci added a comment.Aug 7 2020, 12:32 PM

Build is green

Patch application report for D3728 (id=13158)

Rebasing onto 243e873175...

Current branch diff-target is up to date.
Changes applied before test
commit bfa8f46ea44d05d5dce15071d164b368da3f975f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Aug 6 22:56:09 2020 +0200

    storage*: Rename content_get_data(Sha1) -> Optional[bytes]
    
    Rename the confusing endpoint `content_get` to `content_get_data`.
    This now works on one content as it is how it is used today.
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/714/ for more details.

Harbormaster completed remote builds in B14425: Diff 13158.Aug 7 2020, 12:32 PM
Closed by commit rDSTObfa8f46ea44d: storage*: Rename content_get_data(Sha1) -> Optional[bytes] (authored by ardumont). · Explain WhyAug 7 2020, 12:35 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDSTObfa8f46ea44d: storage*: Rename content_get_data(Sha1) -> Optional[bytes].

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
storage/
cassandra/
14 lines95%
Loading...
13 lines98%
Loading...
28 lines69%
Loading...
26 lines88%
Loading...
13 lines94%
Loading...
tests/
6 lines100%
Loading...
16 lines100%
Loading...
42 lines99%
Loading...

Diff 13158

View Options

swh/storage/cassandra/storage.py

Loading...
View Options

swh/storage/in_memory.py

Loading...
View Options

swh/storage/interface.py

Loading...
View Options

swh/storage/objstorage.py

Loading...
View Options

swh/storage/storage.py

Loading...
View Options

swh/storage/tests/test_filter.py

Loading...
View Options

swh/storage/tests/test_retry.py

Loading...
View Options

swh/storage/tests/test_storage.py

Loading...
About · Developer information · Legal