Page MenuHomeSoftware Heritage

algos.origin: Simplify origin_get_latest_visit_status function
ClosedPublic

Authored by ardumont on Jul 27 2020, 8:17 AM.

Details

Summary

And use storage*.origin_visit_get_latest

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
refactor-algos-origin
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13969
Build 21437: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 21436: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D3621 (id=12731)

Could not rebase; Attempt merge onto 789972fd5f...

Auto-merging swh/storage/tests/test_storage.py
Auto-merging swh/storage/storage.py
Merge made by the 'recursive' strategy.
 swh/storage/algos/origin.py              |  47 +++---
 swh/storage/cassandra/storage.py         |  11 +-
 swh/storage/in_memory.py                 |  38 +++--
 swh/storage/interface.py                 |  15 +-
 swh/storage/storage.py                   |  11 +-
 swh/storage/tests/algos/test_snapshot.py |   8 +-
 swh/storage/tests/test_storage.py        | 268 ++++++++++++++-----------------
 7 files changed, 184 insertions(+), 214 deletions(-)
Changes applied before test
commit 6a68eac113d1670ca7933975d7a34e1f209930c4
Merge: 789972fd 3f0d5040
Author: Jenkins user <jenkins@localhost>
Date:   Mon Jul 27 06:23:39 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 3f0d50403226a51a457401eb654cd4fb961a72b8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 10:22:40 2020 +0200

    algos.origin: Simplify origin_get_latest_visit_status function

commit 8bc0a757b05ba7b9cbdaacbd0c43b175992121cd
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 08:05:10 2020 +0200

    storage*: type origin_visit_get_latest endpoint result
    
    The endpoint returns an optional OriginVisit object instead of a dict:
def origin_visit_get_latest(...) -> Optional[OriginVisit]
```

Note that it also fixes the in-memory storage implementation which filtered
data too early.

Related to T645
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/561/ for more details.
This revision is now accepted and ready to land.Jul 27 2020, 9:44 AM

Build is green

Patch application report for D3621 (id=12739)

Could not rebase; Attempt merge onto 789972fd5f...

Auto-merging swh/storage/tests/test_storage.py
Auto-merging swh/storage/storage.py
Merge made by the 'recursive' strategy.
 swh/storage/algos/origin.py              |  47 ++---
 swh/storage/cassandra/storage.py         |  11 +-
 swh/storage/in_memory.py                 |  38 ++--
 swh/storage/interface.py                 |  15 +-
 swh/storage/storage.py                   |  11 +-
 swh/storage/tests/algos/test_snapshot.py |   8 +-
 swh/storage/tests/test_storage.py        | 323 +++++++++++++++----------------
 7 files changed, 224 insertions(+), 229 deletions(-)
Changes applied before test
commit c6619e68fd01d8d6d1371c758503080c5bcc0636
Merge: 789972fd 7fe12cc7
Author: Jenkins user <jenkins@localhost>
Date:   Mon Jul 27 09:48:54 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 7fe12cc7b2bf94bed4699015170a6ad70bd138f2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 10:22:40 2020 +0200

    algos.origin: Simplify origin_get_latest_visit_status function

commit 10d74885f1866dc493a076df3169977ba57749d6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 08:05:10 2020 +0200

    storage*: type origin_visit_get_latest endpoint result
    
    The endpoint returns an optional OriginVisit object instead of a dict:
def origin_visit_get_latest(...) -> Optional[OriginVisit]
```

It also fixes the in-memory storage implementation which filtered data too
early. It only filtered on the latest origin visit status associated to the
origin visit. So depending on filters, this could have been wrong. It was not
much of a problem as there is no longer any direct clients of this api (they
are using [1] now).

[1] swh.storage.algos.origin.origin_get_latest_origin_visit_status function

Related to T645
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/564/ for more details.

Build is green

Patch application report for D3621 (id=12742)

Could not rebase; Attempt merge onto 789972fd5f...

Updating 789972fd..b31c3049
Fast-forward
 swh/storage/algos/origin.py              |  47 ++---
 swh/storage/cassandra/storage.py         |  11 +-
 swh/storage/in_memory.py                 |  38 ++--
 swh/storage/interface.py                 |  15 +-
 swh/storage/storage.py                   |  11 +-
 swh/storage/tests/algos/test_snapshot.py |   8 +-
 swh/storage/tests/test_storage.py        | 323 +++++++++++++++----------------
 7 files changed, 224 insertions(+), 229 deletions(-)
Changes applied before test
commit b31c30491e536b29bc0552c3ab2c6854b8aaf7d1
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 10:22:40 2020 +0200

    algos.origin: Simplify origin_get_latest_visit_status function

commit 5344a6ff9c40290e670de9fd0d71c0b84185e4fa
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 26 08:05:10 2020 +0200

    storage*: type origin_visit_get_latest endpoint result
    
    The endpoint returns an optional OriginVisit object instead of a dict:
def origin_visit_get_latest(...) -> Optional[OriginVisit]
```

It also fixes the in-memory storage implementation which filtered data too
early. It only filtered on the latest origin visit status associated to the
origin visit. So depending on filters, this could have been wrong. It was not
much of a problem as there is no longer any direct clients of this api (they
are using [1] now).

[1] swh.storage.algos.origin.origin_get_latest_origin_visit_status function

Related to T645
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/566/ for more details.