Page MenuHomeSoftware Heritage
Differential D3621

algos.origin: Simplify origin_get_latest_visit_status function
ClosedPublic
Actions

Authored by ardumont on Jul 27 2020, 8:17 AM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDSTOb31c30491e53: algos.origin: Simplify origin_get_latest_visit_status function
Summary

And use storage*.origin_visit_get_latest

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Jul 27 2020, 8:17 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptJul 27 2020, 8:17 AM
swh-public-ci added a comment.Jul 27 2020, 8:31 AM

Build is green

Patch application report for D3621 (id=12731)

Could not rebase; Attempt merge onto 789972fd5f...

Auto-merging swh/storage/tests/test_storage.py
Auto-merging swh/storage/storage.py
Merge made by the 'recursive' strategy.
 swh/storage/algos/origin.py              |  47 +++---
 swh/storage/cassandra/storage.py         |  11 +-
 swh/storage/in_memory.py                 |  38 +++--
 swh/storage/interface.py                 |  15 +-
 swh/storage/storage.py                   |  11 +-
 swh/storage/tests/algos/test_snapshot.py |   8 +-
 swh/storage/tests/test_storage.py        | 268 ++++++++++++++-----------------
 7 files changed, 184 insertions(+), 214 deletions(-)
Changes applied before test
commit 6a68eac113d1670ca7933975d7a34e1f209930c4
Merge: 789972fd 3f0d5040
Author: Jenkins user <jenkins@localhost>
Date:   Mon Jul 27 06:23:39 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 3f0d50403226a51a457401eb654cd4fb961a72b8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 10:22:40 2020 +0200

    algos.origin: Simplify origin_get_latest_visit_status function

commit 8bc0a757b05ba7b9cbdaacbd0c43b175992121cd
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 08:05:10 2020 +0200

    storage*: type origin_visit_get_latest endpoint result
    
    The endpoint returns an optional OriginVisit object instead of a dict:
def origin_visit_get_latest(...) -> Optional[OriginVisit]
```

Note that it also fixes the in-memory storage implementation which filtered
data too early.

Related to T645
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/561/ for more details.
Harbormaster completed remote builds in B13961: Diff 12731.Jul 27 2020, 8:31 AM
vlorentz accepted this revision.Jul 27 2020, 9:44 AM
vlorentz added a subscriber: vlorentz.

indeed!

This revision is now accepted and ready to land.Jul 27 2020, 9:44 AM
ardumont updated this revision to Diff 12739.Jul 27 2020, 11:43 AM

Rebase

swh-public-ci added a comment.Jul 27 2020, 11:56 AM

Build is green

Patch application report for D3621 (id=12739)

Could not rebase; Attempt merge onto 789972fd5f...

Auto-merging swh/storage/tests/test_storage.py
Auto-merging swh/storage/storage.py
Merge made by the 'recursive' strategy.
 swh/storage/algos/origin.py              |  47 ++---
 swh/storage/cassandra/storage.py         |  11 +-
 swh/storage/in_memory.py                 |  38 ++--
 swh/storage/interface.py                 |  15 +-
 swh/storage/storage.py                   |  11 +-
 swh/storage/tests/algos/test_snapshot.py |   8 +-
 swh/storage/tests/test_storage.py        | 323 +++++++++++++++----------------
 7 files changed, 224 insertions(+), 229 deletions(-)
Changes applied before test
commit c6619e68fd01d8d6d1371c758503080c5bcc0636
Merge: 789972fd 7fe12cc7
Author: Jenkins user <jenkins@localhost>
Date:   Mon Jul 27 09:48:54 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 7fe12cc7b2bf94bed4699015170a6ad70bd138f2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 10:22:40 2020 +0200

    algos.origin: Simplify origin_get_latest_visit_status function

commit 10d74885f1866dc493a076df3169977ba57749d6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 08:05:10 2020 +0200

    storage*: type origin_visit_get_latest endpoint result
    
    The endpoint returns an optional OriginVisit object instead of a dict:
def origin_visit_get_latest(...) -> Optional[OriginVisit]
```

It also fixes the in-memory storage implementation which filtered data too
early. It only filtered on the latest origin visit status associated to the
origin visit. So depending on filters, this could have been wrong. It was not
much of a problem as there is no longer any direct clients of this api (they
are using [1] now).

[1] swh.storage.algos.origin.origin_get_latest_origin_visit_status function

Related to T645
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/564/ for more details.
Harbormaster completed remote builds in B13969: Diff 12739.Jul 27 2020, 11:56 AM
ardumont updated this revision to Diff 12742.Jul 27 2020, 11:56 AM

Rebase

swh-public-ci added a comment.Jul 27 2020, 12:10 PM

Build is green

Patch application report for D3621 (id=12742)

Could not rebase; Attempt merge onto 789972fd5f...

Updating 789972fd..b31c3049
Fast-forward
 swh/storage/algos/origin.py              |  47 ++---
 swh/storage/cassandra/storage.py         |  11 +-
 swh/storage/in_memory.py                 |  38 ++--
 swh/storage/interface.py                 |  15 +-
 swh/storage/storage.py                   |  11 +-
 swh/storage/tests/algos/test_snapshot.py |   8 +-
 swh/storage/tests/test_storage.py        | 323 +++++++++++++++----------------
 7 files changed, 224 insertions(+), 229 deletions(-)
Changes applied before test
commit b31c30491e536b29bc0552c3ab2c6854b8aaf7d1
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 10:22:40 2020 +0200

    algos.origin: Simplify origin_get_latest_visit_status function

commit 5344a6ff9c40290e670de9fd0d71c0b84185e4fa
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 08:05:10 2020 +0200

    storage*: type origin_visit_get_latest endpoint result
    
    The endpoint returns an optional OriginVisit object instead of a dict:
def origin_visit_get_latest(...) -> Optional[OriginVisit]
```

It also fixes the in-memory storage implementation which filtered data too
early. It only filtered on the latest origin visit status associated to the
origin visit. So depending on filters, this could have been wrong. It was not
much of a problem as there is no longer any direct clients of this api (they
are using [1] now).

[1] swh.storage.algos.origin.origin_get_latest_origin_visit_status function

Related to T645
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/566/ for more details.
Harbormaster completed remote builds in B13972: Diff 12742.Jul 27 2020, 12:10 PM
Closed by commit rDSTOb31c30491e53: algos.origin: Simplify origin_get_latest_visit_status function (authored by ardumont). · Explain WhyJul 27 2020, 12:13 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDSTOb31c30491e53: algos.origin: Simplify origin_get_latest_visit_status function.

Revision Contents
Changeset List

PathSize
swh/
storage/
algos/
47 lines

Diff 12749

View Options

swh/storage/algos/origin.py

Loading...
About · Developer information · Legal