Paths

Table of Contentst

Differential D3605

storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
ClosedPublic
Actions

Authored by ardumont on Jul 23 2020, 5:54 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T645: Type swh-storage endpoints with swh.model objects

Commits

rDSTO7e94767a51a7: storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]

Summary

This:

drops the legacy behavior (no more input as list of dicts or even one dict).
aligns with other _get endpoints (only 1 iterable of identifiers as input, here the origin urls).
migrates towards returning an iterable of optional origin model objects (again the optional part is alignment with existing get endpoint)

Impacts foreseen, mostly tests (but not only):

$ grep -r 'origin_get(' */swh/** | grep -v "swh-storage"
swh-deposit/swh/deposit/migrations/0018_migrate_swhids.py:        check_origin = storage.origin_get({"url": origin})
swh-deposit/swh/deposit/migrations/0018_migrate_swhids.py:        check_origin = storage.origin_get({"url": origin})
swh-loader-git/swh/loader/git/loader.py:            base_origin = self.storage.origin_get(base_origin)
swh-loader-svn/swh/loader/svn/loader.py:            origin = self.storage.origin_get({"url": svn_url})
swh-web/swh/web/common/service.py:        origin_info = storage.origin_get(orig)
swh-web/swh/web/common/service.py:        origin = storage.origin_get({"url": match["id"]})
swh-web/swh/web/common/service.py:    origin_info = storage.origin_get(origin_dict)
swh-web/swh/web/tests/api/views/test_origin.py:    expected_origin = archive_data.origin_get(origin)
swh-web/swh/web/tests/data.py:        origin.update(storage.origin_get(origin))  # add an 'id' key if enabled
swh-web/swh/web/tests/common/test_service.py:    expected_origin = archive_data.origin_get({"url": new_origin.url})
swh-web/swh/web/tests/conftest.py:    def origin_get(self, origin_info):
swh-web/swh/web/tests/conftest.py:        origin = self.storage.origin_get(origin_info)
swh-web/swh/web/tests/strategies.py:        lambda origin: get_tests_data()["storage"].origin_get([origin.to_dict()])[0]

Related to T645

Test Plan

tox

Diff Detail

Repository

rDSTO Storage manager

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 13922
Build 21359: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 21358: arc lint + arc unit

Event Timeline

ardumont created this revision.Jul 23 2020, 5:54 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 23 2020, 5:54 PM

Build is green

Patch application report for D3605 (id=12696)

Rebasing onto d8583eb468...

Current branch diff-target is up to date.

Changes applied before test

commit 94ebfe37ebaa02eda8ab0aab0afea274d6ec2baf
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 23 17:47:04 2020 +0200

    storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/555/ for more details.

Harbormaster completed remote builds in B13921: Diff 12696.Jul 23 2020, 6:02 PM

vlorentz accepted this revision.Jul 23 2020, 6:06 PM

vlorentz added a subscriber: vlorentz.

vlorentz added inline comments.

swh/storage/interface.py
956	shouldn't have removed that line

This revision is now accepted and ready to land.Jul 23 2020, 6:06 PM

Fix docstring typo

Rework commit message

Build is green

Patch application report for D3605 (id=12697)

Rebasing onto d8583eb468...

Current branch diff-target is up to date.

Changes applied before test

commit bca6fe7256eecc415bcc67f485dbde1c5e78780b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 23 17:47:04 2020 +0200

    storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/556/ for more details.

Harbormaster completed remote builds in B13922: Diff 12697.Jul 23 2020, 6:22 PM

Build is green

Patch application report for D3605 (id=12698)

Rebasing onto d8583eb468...

Current branch diff-target is up to date.

Changes applied before test

commit 7e94767a51a743325e1151ff878b6e3b78622ef7
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 23 17:47:04 2020 +0200

    storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]]
    
    This:
    - drops the legacy behavior (no more input as list of dicts or even one dict).
    - aligns with other _get endpoints (only 1 iterable of identifiers as input,
    here the origin urls).
    - migrates towards returning an iterable of optional origin model
    objects (again the optional part is alignment with existing get endpoint)
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/557/ for more details.

Harbormaster completed remote builds in B13923: Diff 12698.Jul 23 2020, 6:28 PM

Closed by commit rDSTO7e94767a51a7: storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]] (authored by ardumont). · Explain WhyJul 24 2020, 8:08 AM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDSTO7e94767a51a7: storage*: origin_get(Iterable[str]) -> Iterable[Optional[Origin]].

ardumont mentioned this in D3608: loader: Update swh.storage.origin_get call to latest api change.Jul 24 2020, 9:30 AM

ardumont mentioned this in D3609: loader: Update swh.storage.origin_get call to latest api change.Jul 24 2020, 9:32 AM