Differential D3863

Refactor revision_get storage API to return Revision objects
ClosedPublic
Actions

Authored by ardumont on Sep 2 2020, 3:25 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T645: Type swh-storage endpoints with swh.model objects

Commits

rDSTO356eacd763d6: Refactor revision_get storage API to return Revision objects

Summary

The signature becomes revision_get(...) -> List[Optional[Revision]]

Impacts:

D3864: swh-deposit
D3865: swh-indexer
D3868: swh-loader-core
D3869: swh-loader-mercurial
D3870: swh-loader-svn
D3877: swh-web

$ grep -r "revision_get" $SWH_ENVIRONMENT_HOME/*/swh/** | grep -v swh/storage | grep revision_get
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/migrations/0018_migrate_swhids.py:        revisions = list(storage.revision_get([rev_id_bytes]))
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/indexer.py:        for rev in self.storage.revision_get(revision_ids):
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/metadata.py:                self.storage.revision_get, head_rev_ids, REVISION_GET_BATCH_SIZE
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/tests/__init__.py:        revisions = list(storage.revision_get(revs))
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/pypi/tests/test_pypi.py:    revision = list(loader.storage.revision_get([expected_revision_id]))[0]
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/nixguix/loader.py:        known_revisions = self.storage.revision_get(revs)
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/nixguix/loader.py:            if not revision:  # revision_get can return None
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/nixguix/tests/test_nixguix.py:    revisions = storage.revision_get(revision_ids)
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/nixguix/tests/test_nixguix.py:        old_revision = next(loader.storage.revision_get([first_revision.target]))
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/nixguix/tests/test_nixguix.py:        new_revision = next(loader.storage.revision_get([new_revision_branch.target]))
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/archive/tests/test_archive.py:    revision = list(loader.storage.revision_get([expected_revision_id]))[0]
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/loader.py:        known_revisions = self.storage.revision_get(revs)
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/deposit/loader.py:            revisions = list(self.storage.revision_get([rev_id]))
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/deposit/tests/test_deposit.py:    revision = list(loader.storage.revision_get([expected_revision_id]))[0]
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/deposit/tests/test_deposit.py:    revision = next(loader.storage.revision_get([revision_id]))
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/deposit/tests/test_deposit.py:    revision = next(loader.storage.revision_get([hash_to_bytes(revision_id)]))
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/npm/tests/test_npm.py:    revision = list(loader.storage.revision_get([expected_revision_id]))[0]
$SWH_ENVIRONMENT_HOME/swh-loader-mercurial/swh/loader/mercurial/tests/test_loader.py:    revision = loader.storage.revision_get([tip_revision_default])
$SWH_ENVIRONMENT_HOME/swh-loader-svn/swh/loader/svn/loader.py:        revs = list(storage.revision_get([swh_id]))
$SWH_ENVIRONMENT_HOME/swh-loader-svn/swh/loader/svn/tests/test_loader.py:        loader.storage.revision_get(
$SWH_ENVIRONMENT_HOME/swh-loader-svn/swh/loader/svn/tests/test_loader.py:        loader.storage.revision_get(
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:    revision = _first_element(storage.revision_get([sha1_git_bin]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:    revisions = storage.revision_get(sha1_bin_list) or []
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:    revision = _first_element(storage.revision_get([sha1_git_bin]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:    rev_root = _first_element(storage.revision_get([rev_root_id_bin]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:    revision = _first_element(storage.revision_get([sha1_git_bin]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:        revision_root = _first_element(storage.revision_get([sha1_git_root_bin]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:    revision = _first_element(storage.revision_get([sha1_git_bin]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/common/service.py:        revision = next(storage.revision_get([entity["target"]]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_origin.py:        head_rev = archive_data.revision_get(head_rev_id)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_origin.py:    head_rev = archive_data.revision_get(head_rev_id)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_origin.py:    head_rev = archive_data.revision_get(head_rev_id)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_origin.py:    revision_data = archive_data.revision_get(release_data["target"])
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_revision.py:    revision_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_content.py:    directory = archive_data.revision_get(branch_info["revision"])["directory"]
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_release.py:            rev = archive_data.revision_get(release_data["target"])
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_identifiers.py:    directory = archive_data.revision_get(revision)["directory"]
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/views/test_directory.py:    directory = archive_data.revision_get(branch_info["revision"])["directory"]
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/test_snapshot_context.py:                rev_data = archive_data.revision_get(branch_data["target"])
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/test_snapshot_context.py:                rev_data = archive_data.revision_get(rel_data["target"])
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/browse/test_snapshot_context.py:        revision_info = archive_data.revision_get(revision_id)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/api/views/test_revision.py:    expected_revision = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/api/views/test_revision.py:    expected_message = archive_data.revision_get(revision)["message"]
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/api/test_utils.py:    revision_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/api/test_utils.py:    revision_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/api/test_utils.py:    revision_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/api/test_utils.py:    revision_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/data.py:        for rev in storage.revision_get(revisions):
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_identifiers.py:    revision_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_identifiers.py:        head_rev = archive_data.revision_get(head_rev_id)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_identifiers.py:    directory = archive_data.revision_get(branch_info["revision"])["directory"]
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:        expected_revision = archive_data.revision_get(sha1_git)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    revision = archive_data.revision_get(revision_with_submodules["rev_sha1_git"])
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:        "content": archive_data.revision_get(rev_entry["target"]),
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    revision_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    rev_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    rev_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    assert actual_revision == archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:        expected_revisions.append(archive_data.revision_get(rev))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    expected_revision = archive_data.revision_get(branches[branch_name]["target"])
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    expected_root_rev = archive_data.revision_get(root_rev)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    expected_rev = archive_data.revision_get(rev)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    rev_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    rev_data = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/common/test_service.py:    expected = archive_data.revision_get(revision)
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/conftest.py:    def revision_get(self, rev_id):
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/conftest.py:        rev_data = next(self.storage.revision_get([rev_id_bytes]))
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/strategies.py:        lambda s: next(get_tests_data()["storage"].revision_get([hash_to_bytes(s)]))

Related to T645

Test Plan

tox

Diff Detail

Repository

rDSTO Storage manager

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Sep 2 2020, 3:25 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptSep 2 2020, 3:25 PM

Build is green

Patch application report for D3863 (id=13649)

Rebasing onto e6fcfb931a...

Current branch diff-target is up to date.

Changes applied before test

commit 558a61af89c8235b8fa068df34d2284c7df2ff45
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Sep 2 15:18:07 2020 +0200

    storage*: revision_get(...) -> List[Optional[Revision]]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/897/ for more details.

Harbormaster completed remote builds in B14883: Diff 13649.Sep 2 2020, 3:29 PM

ardumont edited the summary of this revision. (Show Details)Sep 2 2020, 3:51 PM

ardumont edited the summary of this revision. (Show Details)Sep 2 2020, 4:08 PM

looks ok but I find the commit message too cryptic.

I'd prefer sometinhg like "Refactor the revision_get() storage API to return Revision objects"

I know it's consistent with previous commit, but still...

ardumont edited the summary of this revision. (Show Details)Sep 2 2020, 6:42 PM

ardumont retitled this revision from storage*: revision_get(...) -> List[Optional[Revision]] to Refactor revision_get storage API to return Revision objects.Sep 3 2020, 10:06 AM

ardumont edited the summary of this revision. (Show Details)

Rework commit message

Build is green

Patch application report for D3863 (id=13659)

Rebasing onto 36d284c730...

First, rewinding head to replay your work on top of it...
Applying: Refactor revision_get storage API to return Revision objects

Changes applied before test

commit 197b5d59e21e671f80e1f717a5b4fc6f548dc11b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Sep 2 15:18:07 2020 +0200

    Refactor revision_get storage API to return Revision objects
    
    The signature becomes revision_get(...) -> List[Optional[Revision]]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/899/ for more details.

Harbormaster completed remote builds in B14892: Diff 13659.Sep 3 2020, 10:11 AM

ardumont edited the summary of this revision. (Show Details)Sep 3 2020, 1:20 PM

ardumont edited the summary of this revision. (Show Details)Sep 3 2020, 1:26 PM

vlorentz accepted this revision.Sep 3 2020, 2:04 PM

This revision is now accepted and ready to land.Sep 3 2020, 2:04 PM

Rebase

Build is green

Patch application report for D3863 (id=13673)

Rebasing onto 36d284c730...

Current branch diff-target is up to date.

Changes applied before test

commit 356eacd763d6a36ad9feaa93ea4d74b3e9ce7a60
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Sep 2 15:18:07 2020 +0200

    Refactor revision_get storage API to return Revision objects
    
    The signature becomes revision_get(...) -> List[Optional[Revision]]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/902/ for more details.

Harbormaster completed remote builds in B14904: Diff 13673.Sep 3 2020, 5:09 PM

Closed by commit rDSTO356eacd763d6: Refactor revision_get storage API to return Revision objects (authored by ardumont). · Explain WhySep 3 2020, 5:15 PM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDSTO356eacd763d6: Refactor revision_get storage API to return Revision objects.

ardumont edited the summary of this revision. (Show Details)Sep 3 2020, 5:49 PM