Page MenuHomeSoftware Heritage

storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
ClosedPublic

Authored by ardumont on Thu, Jul 23, 5:54 PM.

Details

Summary

This:

  • drops the legacy behavior (no more input as list of dicts or even one dict).
  • aligns with other _get endpoints (only 1 iterable of identifiers as input, here the origin urls).
  • migrates towards returning an iterable of optional origin model objects (again the optional part is alignment with existing get endpoint)

Impacts foreseen, mostly tests (but not only):

$ grep -r 'origin_get(' */swh/** | grep -v "swh-storage"
swh-deposit/swh/deposit/migrations/0018_migrate_swhids.py:        check_origin = storage.origin_get({"url": origin})
swh-deposit/swh/deposit/migrations/0018_migrate_swhids.py:        check_origin = storage.origin_get({"url": origin})
swh-loader-git/swh/loader/git/loader.py:            base_origin = self.storage.origin_get(base_origin)
swh-loader-svn/swh/loader/svn/loader.py:            origin = self.storage.origin_get({"url": svn_url})
swh-web/swh/web/common/service.py:        origin_info = storage.origin_get(orig)
swh-web/swh/web/common/service.py:        origin = storage.origin_get({"url": match["id"]})
swh-web/swh/web/common/service.py:    origin_info = storage.origin_get(origin_dict)
swh-web/swh/web/tests/api/views/test_origin.py:    expected_origin = archive_data.origin_get(origin)
swh-web/swh/web/tests/data.py:        origin.update(storage.origin_get(origin))  # add an 'id' key if enabled
swh-web/swh/web/tests/common/test_service.py:    expected_origin = archive_data.origin_get({"url": new_origin.url})
swh-web/swh/web/tests/conftest.py:    def origin_get(self, origin_info):
swh-web/swh/web/tests/conftest.py:        origin = self.storage.origin_get(origin_info)
swh-web/swh/web/tests/strategies.py:        lambda origin: get_tests_data()["storage"].origin_get([origin.to_dict()])[0]

Related to T645

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Thu, Jul 23, 5:54 PM

Build is green

Patch application report for D3605 (id=12696)

Rebasing onto d8583eb468...

Current branch diff-target is up to date.
Changes applied before test
commit 94ebfe37ebaa02eda8ab0aab0afea274d6ec2baf
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 23 17:47:04 2020 +0200

    storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/555/ for more details.

vlorentz accepted this revision.Thu, Jul 23, 6:06 PM
vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/storage/interface.py
956

shouldn't have removed that line

This revision is now accepted and ready to land.Thu, Jul 23, 6:06 PM
ardumont updated this revision to Diff 12697.Thu, Jul 23, 6:15 PM

Fix docstring typo

ardumont updated this revision to Diff 12698.Thu, Jul 23, 6:16 PM

Rework commit message

Build is green

Patch application report for D3605 (id=12697)

Rebasing onto d8583eb468...

Current branch diff-target is up to date.
Changes applied before test
commit bca6fe7256eecc415bcc67f485dbde1c5e78780b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 23 17:47:04 2020 +0200

    storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/556/ for more details.

Build is green

Patch application report for D3605 (id=12698)

Rebasing onto d8583eb468...

Current branch diff-target is up to date.
Changes applied before test
commit 7e94767a51a743325e1151ff878b6e3b78622ef7
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 23 17:47:04 2020 +0200

    storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
    
    This:
    - drops the legacy behavior (no more input as list of dicts or even one dict).
    - aligns with other _get endpoints (only 1 iterable of identifiers as input,
    here the origin urls).
    - migrates towards returning an iterable of optional origin model
    objects (again the optional part is alignment with existing get endpoint)
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/557/ for more details.