Page MenuHomeSoftware Heritage
Differential D3707

storage*: revision_*: Type remaining existing endpoints
ClosedPublic
Actions

Authored by ardumont on Aug 5 2020, 8:38 AM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Maniphest Tasks
T2517: Add remaining missing types to swh.storage.interface
Commits
rDSTO760cbf6db540: storage*: revision_*log: Type remaining existing endpoints
rDSTO38ee5255b0e0: storage*: revision_get: Type remaining existing endpoints
rDSTO8b6d18ef2f9e: storage*: revision_missing: Type remaining existing endpoints
Summary

endpoints:

  • revision_missing
  • revision_get
  • revision_log
  • revision_shortlog

Note: 4 commits in diffs

Related to T645

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 14290
Build 21981: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 21980: arc lint + arc unit

Unit TestsFailed
View All

TimeTest
312 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_cassandra.TestCassandraStorage::test_types
self = <swh.storage.tests.test_cassandra.TestCassandraStorage object at 0x7f836c75eb38> swh_storage_backend_config = {'cls': 'cassandra', 'hosts': ['127.0.0.1'], 'journal_writer': {'cls': 'memory'}, 'keyspace': '0e12f6ff15709e729436', ...}
35 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_in_memory.TestStorage::test_types
self = <swh.storage.tests.test_storage.TestStorage object at 0x7f81cf33e6a0> swh_storage_backend_config = {'cls': 'memory', 'journal_writer': {'cls': 'memory'}}
100 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_storage.TestStorage::test_revision_log
self = <swh.storage.tests.test_storage.TestStorage object at 0x7f829069fc18> swh_storage = <swh.storage.storage.Storage object at 0x7f81ea5f7668> sample_data = <swh.storage.tests.storage_data.StorageData object at 0x7f827824d320>
101 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_storage.TestStorage::test_revision_log_with_limit
self = <swh.storage.tests.test_storage.TestStorage object at 0x7f81ca739630> swh_storage = <swh.storage.storage.Storage object at 0x7f81ca739748> sample_data = <swh.storage.tests.storage_data.StorageData object at 0x7f81ca6dfac8>
3 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_content
View Full Test Results (4 Failed · 728 Passed · 15 Skipped)

Event Timeline

ardumont created this revision.Aug 5 2020, 8:38 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptAug 5 2020, 8:38 AM
swh-public-ci added a comment.Aug 5 2020, 8:43 AM

Build is green

Patch application report for D3707 (id=13046)

Could not rebase; Attempt merge onto a5232b7a7b...

Updating a5232b7a..fbda52b3
Fast-forward
 swh/storage/cassandra/converters.py |   2 +-
 swh/storage/cassandra/storage.py    | 110 +++++++++++++++--------
 swh/storage/converters.py           |   6 +-
 swh/storage/filter.py               |  19 ++--
 swh/storage/in_memory.py            | 107 +++++++++++++++--------
 swh/storage/interface.py            | 168 ++++++++++++++++++++----------------
 swh/storage/storage.py              | 104 +++++++++++++---------
 7 files changed, 314 insertions(+), 202 deletions(-)
Changes applied before test
commit fbda52b3a3e7819a042e98715285b175637a413b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:37:43 2020 +0200

    storage*: revision_shortlog: Type remaining existing endpoints
    
    Related to T645

commit 4c31534b2d5b4da90f31c62d5f9cb9ef6d41090d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:33:49 2020 +0200

    storage*: revision_log: Type remaining existing endpoints
    
    Related to T645

commit 38ee5255b0e0e34a31a7e154a9aac99c9ed16de9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:17:42 2020 +0200

    storage*: revision_get: Type remaining existing endpoints
    
    Related to T645

commit 8b6d18ef2f9e8834906229b16a367c00fb0b95b2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:11:04 2020 +0200

    storage*: revision_missing: Type remaining existing endpoints
    
    Related to T645

commit 9f214bc745ee666071384b47da8a2f6cc7ab65da
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 23:04:26 2020 +0200

    storage*: directory_entry_get_by_path: Type remaining existing endpoints
    
    Related to T645

commit f9d09527a6ba682c7cdb487de3538fb58fa8b346
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:35:45 2020 +0200

    storage*: directory_ls: Type remaining existing endpoints
    
    Related to T645

commit fd5fd86b11a8dafeb564a3d07624becf57d90672
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:27:45 2020 +0200

    storage*: directory_missing: Type remaining existing endpoints
    
    Related to T645

commit 5d13cd7c545141a632e6e60e485cdc33e451f4e2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:07:34 2020 +0200

    storage*: skipped_content_missing: Type remaining existing endpoints
    
    Related to T645

commit 1a2aa70c687cbdac7655257ac3e6e572c8da43b4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:57:37 2020 +0200

    storage*: content_missing_per_sha1_git: Type remaining existing endpoints
    
    Related to T645

commit 15e48633a6a7beeccd233a38cdd456070fb4ae07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:55:39 2020 +0200

    storage*: content_missing_per_sha1: Type remaining existing endpoints
    
    Related to T645

commit b62afbbbdd95aaa91163c8907fa7237231d7c3a6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:13:43 2020 +0200

    storage*: content_missing: Unify and type remaining existing endpoints
    
    This updates the docstrings as well
    
    Related to T645

commit d6f26e45e9a62894ab2b888a15f8a4417c3b2ce0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:06:06 2020 +0200

    storage*: content_get_partition: Type remaining existing endpoints
    
    Related to T645

commit 864473370acdfd5acaf9afbfea9d16745b3c102f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:56:39 2020 +0200

    storage*: content_get_range: Type remaining existing endpoints
    
    Related to T645

commit c6da28289ec4f1be1eb1c1e78d561b8cc8dac937
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:45:31 2020 +0200

    storage*: content_get: Type remaining existing endpoints
    
    Related to T645

commit 25ebc48198d665cc3bb71c96a8ae54775ddf38a8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:42:12 2020 +0200

    storage*: content_update: Type remaining existing endpoints
    
    Related to T645

commit c32e224d23996705a6ca969696cc4715598ed796
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

commit 26ef01563f058bdfe77fcf5d2ca6a897e2894bc8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:24 2020 +0200

    storage*: check_config: Type remaining existing endpoints
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/683/ for more details.

Harbormaster completed remote builds in B14289: Diff 13046.Aug 5 2020, 8:43 AM
ardumont updated this revision to Diff 13047.Aug 5 2020, 8:53 AM

Fix revision_log signature on storage

swh-public-ci added a comment.Aug 5 2020, 8:58 AM

Build has FAILED

Patch application report for D3707 (id=13047)

Could not rebase; Attempt merge onto a5232b7a7b...

Updating a5232b7a..c5c4e5c8
Fast-forward
 swh/storage/cassandra/converters.py |   2 +-
 swh/storage/cassandra/storage.py    | 110 +++++++++++++++--------
 swh/storage/converters.py           |   6 +-
 swh/storage/filter.py               |  19 ++--
 swh/storage/in_memory.py            | 107 +++++++++++++++--------
 swh/storage/interface.py            | 168 ++++++++++++++++++++----------------
 swh/storage/storage.py              |  98 +++++++++++++--------
 7 files changed, 313 insertions(+), 197 deletions(-)
Changes applied before test
commit c5c4e5c8ada3ac18d66891d9b404a11d34939dfa
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:37:43 2020 +0200

    storage*: revision_shortlog: Type remaining existing endpoints
    
    Related to T645

commit 3bcf016c5f0453dca015e628d140fba303f95e74
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:33:49 2020 +0200

    storage*: revision_log: Type remaining existing endpoints
    
    Related to T645

commit 38ee5255b0e0e34a31a7e154a9aac99c9ed16de9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:17:42 2020 +0200

    storage*: revision_get: Type remaining existing endpoints
    
    Related to T645

commit 8b6d18ef2f9e8834906229b16a367c00fb0b95b2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:11:04 2020 +0200

    storage*: revision_missing: Type remaining existing endpoints
    
    Related to T645

commit 9f214bc745ee666071384b47da8a2f6cc7ab65da
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 23:04:26 2020 +0200

    storage*: directory_entry_get_by_path: Type remaining existing endpoints
    
    Related to T645

commit f9d09527a6ba682c7cdb487de3538fb58fa8b346
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:35:45 2020 +0200

    storage*: directory_ls: Type remaining existing endpoints
    
    Related to T645

commit fd5fd86b11a8dafeb564a3d07624becf57d90672
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:27:45 2020 +0200

    storage*: directory_missing: Type remaining existing endpoints
    
    Related to T645

commit 5d13cd7c545141a632e6e60e485cdc33e451f4e2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:07:34 2020 +0200

    storage*: skipped_content_missing: Type remaining existing endpoints
    
    Related to T645

commit 1a2aa70c687cbdac7655257ac3e6e572c8da43b4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:57:37 2020 +0200

    storage*: content_missing_per_sha1_git: Type remaining existing endpoints
    
    Related to T645

commit 15e48633a6a7beeccd233a38cdd456070fb4ae07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:55:39 2020 +0200

    storage*: content_missing_per_sha1: Type remaining existing endpoints
    
    Related to T645

commit b62afbbbdd95aaa91163c8907fa7237231d7c3a6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:13:43 2020 +0200

    storage*: content_missing: Unify and type remaining existing endpoints
    
    This updates the docstrings as well
    
    Related to T645

commit d6f26e45e9a62894ab2b888a15f8a4417c3b2ce0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:06:06 2020 +0200

    storage*: content_get_partition: Type remaining existing endpoints
    
    Related to T645

commit 864473370acdfd5acaf9afbfea9d16745b3c102f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:56:39 2020 +0200

    storage*: content_get_range: Type remaining existing endpoints
    
    Related to T645

commit c6da28289ec4f1be1eb1c1e78d561b8cc8dac937
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:45:31 2020 +0200

    storage*: content_get: Type remaining existing endpoints
    
    Related to T645

commit 25ebc48198d665cc3bb71c96a8ae54775ddf38a8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:42:12 2020 +0200

    storage*: content_update: Type remaining existing endpoints
    
    Related to T645

commit c32e224d23996705a6ca969696cc4715598ed796
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

commit 26ef01563f058bdfe77fcf5d2ca6a897e2894bc8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:24 2020 +0200

    storage*: check_config: Type remaining existing endpoints
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/684/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/684/console

Harbormaster failed remote builds in B14290: Diff 13047!Aug 5 2020, 8:58 AM
ardumont added inline comments.Aug 5 2020, 9:03 AM
swh/storage/cassandra/storage.py
530

i'm a bit confused here.

the pg-storage returns Optional[Dict[str, Any]] but both cass and in-memory don't.

I think those are wrong because they assume the input revisions exist but they could be inexistent.
I think pg-storage implementation is correct because of that.

vlorentz added a subscriber: vlorentz.Aug 5 2020, 9:06 AM
vlorentz added inline comments.
swh/storage/cassandra/storage.py
530

Yes, indeed. The current type of _get_parent_revs is incorrect because it sometimes returns None.

swh/storage/in_memory.py
542

Iterable[Optional[Dict[str, Any]]]

vlorentz requested changes to this revision.Aug 5 2020, 9:07 AM
This revision now requires changes to proceed.Aug 5 2020, 9:07 AM
ardumont added inline comments.Aug 5 2020, 9:17 AM
swh/storage/in_memory.py
542

is return None and yield None in that context are similar?

^ that confuses me

542

is return None and yield None in that context are similar?

(fixing my typo)
Are return None and yield None in that context similar?

ardumont updated this revision to Diff 13048.Aug 5 2020, 9:36 AM

Fix revision_*log endpoints issue

swh-public-ci added a comment.Aug 5 2020, 9:41 AM

Build has FAILED

Patch application report for D3707 (id=13048)

Could not rebase; Attempt merge onto a5232b7a7b...

Updating a5232b7a..fb4c721c
Fast-forward
 swh/storage/cassandra/converters.py |   2 +-
 swh/storage/cassandra/storage.py    | 113 ++++++++++++++++--------
 swh/storage/converters.py           |   6 +-
 swh/storage/filter.py               |  19 ++--
 swh/storage/in_memory.py            | 110 +++++++++++++++--------
 swh/storage/interface.py            | 168 ++++++++++++++++++++----------------
 swh/storage/storage.py              |  98 +++++++++++++--------
 7 files changed, 318 insertions(+), 198 deletions(-)
Changes applied before test
commit fb4c721ce6b99398c54a402fa712219cd80d0265
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:33:49 2020 +0200

    storage*: revision_*log: Type remaining existing endpoints
    
    Related to T645

commit 38ee5255b0e0e34a31a7e154a9aac99c9ed16de9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:17:42 2020 +0200

    storage*: revision_get: Type remaining existing endpoints
    
    Related to T645

commit 8b6d18ef2f9e8834906229b16a367c00fb0b95b2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:11:04 2020 +0200

    storage*: revision_missing: Type remaining existing endpoints
    
    Related to T645

commit 9f214bc745ee666071384b47da8a2f6cc7ab65da
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 23:04:26 2020 +0200

    storage*: directory_entry_get_by_path: Type remaining existing endpoints
    
    Related to T645

commit f9d09527a6ba682c7cdb487de3538fb58fa8b346
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:35:45 2020 +0200

    storage*: directory_ls: Type remaining existing endpoints
    
    Related to T645

commit fd5fd86b11a8dafeb564a3d07624becf57d90672
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:27:45 2020 +0200

    storage*: directory_missing: Type remaining existing endpoints
    
    Related to T645

commit 5d13cd7c545141a632e6e60e485cdc33e451f4e2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:07:34 2020 +0200

    storage*: skipped_content_missing: Type remaining existing endpoints
    
    Related to T645

commit 1a2aa70c687cbdac7655257ac3e6e572c8da43b4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:57:37 2020 +0200

    storage*: content_missing_per_sha1_git: Type remaining existing endpoints
    
    Related to T645

commit 15e48633a6a7beeccd233a38cdd456070fb4ae07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:55:39 2020 +0200

    storage*: content_missing_per_sha1: Type remaining existing endpoints
    
    Related to T645

commit b62afbbbdd95aaa91163c8907fa7237231d7c3a6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:13:43 2020 +0200

    storage*: content_missing: Unify and type remaining existing endpoints
    
    This updates the docstrings as well
    
    Related to T645

commit d6f26e45e9a62894ab2b888a15f8a4417c3b2ce0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:06:06 2020 +0200

    storage*: content_get_partition: Type remaining existing endpoints
    
    Related to T645

commit 864473370acdfd5acaf9afbfea9d16745b3c102f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:56:39 2020 +0200

    storage*: content_get_range: Type remaining existing endpoints
    
    Related to T645

commit c6da28289ec4f1be1eb1c1e78d561b8cc8dac937
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:45:31 2020 +0200

    storage*: content_get: Type remaining existing endpoints
    
    Related to T645

commit 25ebc48198d665cc3bb71c96a8ae54775ddf38a8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:42:12 2020 +0200

    storage*: content_update: Type remaining existing endpoints
    
    Related to T645

commit c32e224d23996705a6ca969696cc4715598ed796
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

commit 26ef01563f058bdfe77fcf5d2ca6a897e2894bc8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:24 2020 +0200

    storage*: check_config: Type remaining existing endpoints
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/685/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/685/console

Harbormaster failed remote builds in B14291: Diff 13048!Aug 5 2020, 9:41 AM
vlorentz added inline comments.Aug 5 2020, 9:44 AM
swh/storage/in_memory.py
542

Oh my bad, I didn't notice it's a generator. Then it should be return, not return None.

ardumont added inline comments.Aug 5 2020, 9:48 AM
swh/storage/in_memory.py
542

heh, don't worry, that's the point of review, discussion and improvments ;)

(i need to fix the build anyway because it's broken somehow... just checking the types test is not enough)

ardumont added inline comments.Aug 5 2020, 9:51 AM
swh/storage/storage.py
635

typo!

ardumont updated this revision to Diff 13049.Aug 5 2020, 9:54 AM
  • Revert return None back to return
  • Revert line back to data (typo introduced when I thought wrongly we could remove that part at some point)
swh-public-ci added a comment.Aug 5 2020, 9:59 AM

Build is green

Patch application report for D3707 (id=13049)

Could not rebase; Attempt merge onto a5232b7a7b...

Updating a5232b7a..a664f682
Fast-forward
 swh/storage/cassandra/converters.py |   2 +-
 swh/storage/cassandra/storage.py    | 113 ++++++++++++++++--------
 swh/storage/converters.py           |   6 +-
 swh/storage/filter.py               |  19 ++--
 swh/storage/in_memory.py            | 106 +++++++++++++++--------
 swh/storage/interface.py            | 168 ++++++++++++++++++++----------------
 swh/storage/storage.py              |  96 +++++++++++++--------
 7 files changed, 315 insertions(+), 195 deletions(-)
Changes applied before test
commit a664f682a50bc498df339309723f064bd69c2510
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:33:49 2020 +0200

    storage*: revision_*log: Type remaining existing endpoints
    
    Related to T645

commit 38ee5255b0e0e34a31a7e154a9aac99c9ed16de9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:17:42 2020 +0200

    storage*: revision_get: Type remaining existing endpoints
    
    Related to T645

commit 8b6d18ef2f9e8834906229b16a367c00fb0b95b2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:11:04 2020 +0200

    storage*: revision_missing: Type remaining existing endpoints
    
    Related to T645

commit 9f214bc745ee666071384b47da8a2f6cc7ab65da
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 23:04:26 2020 +0200

    storage*: directory_entry_get_by_path: Type remaining existing endpoints
    
    Related to T645

commit f9d09527a6ba682c7cdb487de3538fb58fa8b346
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:35:45 2020 +0200

    storage*: directory_ls: Type remaining existing endpoints
    
    Related to T645

commit fd5fd86b11a8dafeb564a3d07624becf57d90672
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:27:45 2020 +0200

    storage*: directory_missing: Type remaining existing endpoints
    
    Related to T645

commit 5d13cd7c545141a632e6e60e485cdc33e451f4e2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:07:34 2020 +0200

    storage*: skipped_content_missing: Type remaining existing endpoints
    
    Related to T645

commit 1a2aa70c687cbdac7655257ac3e6e572c8da43b4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:57:37 2020 +0200

    storage*: content_missing_per_sha1_git: Type remaining existing endpoints
    
    Related to T645

commit 15e48633a6a7beeccd233a38cdd456070fb4ae07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:55:39 2020 +0200

    storage*: content_missing_per_sha1: Type remaining existing endpoints
    
    Related to T645

commit b62afbbbdd95aaa91163c8907fa7237231d7c3a6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:13:43 2020 +0200

    storage*: content_missing: Unify and type remaining existing endpoints
    
    This updates the docstrings as well
    
    Related to T645

commit d6f26e45e9a62894ab2b888a15f8a4417c3b2ce0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:06:06 2020 +0200

    storage*: content_get_partition: Type remaining existing endpoints
    
    Related to T645

commit 864473370acdfd5acaf9afbfea9d16745b3c102f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:56:39 2020 +0200

    storage*: content_get_range: Type remaining existing endpoints
    
    Related to T645

commit c6da28289ec4f1be1eb1c1e78d561b8cc8dac937
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:45:31 2020 +0200

    storage*: content_get: Type remaining existing endpoints
    
    Related to T645

commit 25ebc48198d665cc3bb71c96a8ae54775ddf38a8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:42:12 2020 +0200

    storage*: content_update: Type remaining existing endpoints
    
    Related to T645

commit c32e224d23996705a6ca969696cc4715598ed796
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

commit 26ef01563f058bdfe77fcf5d2ca6a897e2894bc8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:24 2020 +0200

    storage*: check_config: Type remaining existing endpoints
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/686/ for more details.

Harbormaster completed remote builds in B14292: Diff 13049.Aug 5 2020, 9:59 AM
vlorentz accepted this revision.Aug 5 2020, 10:24 AM
vlorentz added inline comments.
swh/storage/in_memory.py
542

You can remove Optional from the signature now

This revision is now accepted and ready to land.Aug 5 2020, 10:24 AM
ardumont added inline comments.Aug 5 2020, 10:33 AM
swh/storage/in_memory.py
542

but if i do that, that won't type check anymore with the remaining part?

(I'm back to my initial confusion)

What did i miss?

vlorentz added inline comments.Aug 5 2020, 10:39 AM
swh/storage/in_memory.py
542

it will, because Iterable[Optional[Dict[str, Any]]] is a supertype of Iterable[Dict[str, Any]]

ardumont updated this revision to Diff 13052.Aug 5 2020, 10:57 AM

Drop unneeded Optional type

swh/storage/in_memory.py
542

I did not realize, thanks.

vlorentz accepted this revision.Aug 5 2020, 11:00 AM
swh-public-ci added a comment.Aug 5 2020, 11:01 AM

Build has FAILED

Patch application report for D3707 (id=13052)

Could not rebase; Attempt merge onto a5232b7a7b...

Updating a5232b7a..a3e39e85
Fast-forward
 swh/storage/cassandra/converters.py |   2 +-
 swh/storage/cassandra/storage.py    | 112 ++++++++++++++++--------
 swh/storage/converters.py           |   6 +-
 swh/storage/filter.py               |  19 ++--
 swh/storage/in_memory.py            | 103 ++++++++++++++--------
 swh/storage/interface.py            | 168 ++++++++++++++++++++----------------
 swh/storage/storage.py              |  96 +++++++++++++--------
 7 files changed, 312 insertions(+), 194 deletions(-)
Changes applied before test
commit a3e39e855ff3712cfcf927caa0f2b8c95dc6b884
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:33:49 2020 +0200

    storage*: revision_*log: Type remaining existing endpoints
    
    Related to T645

commit 38ee5255b0e0e34a31a7e154a9aac99c9ed16de9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:17:42 2020 +0200

    storage*: revision_get: Type remaining existing endpoints
    
    Related to T645

commit 8b6d18ef2f9e8834906229b16a367c00fb0b95b2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:11:04 2020 +0200

    storage*: revision_missing: Type remaining existing endpoints
    
    Related to T645

commit 9f214bc745ee666071384b47da8a2f6cc7ab65da
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 23:04:26 2020 +0200

    storage*: directory_entry_get_by_path: Type remaining existing endpoints
    
    Related to T645

commit f9d09527a6ba682c7cdb487de3538fb58fa8b346
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:35:45 2020 +0200

    storage*: directory_ls: Type remaining existing endpoints
    
    Related to T645

commit fd5fd86b11a8dafeb564a3d07624becf57d90672
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:27:45 2020 +0200

    storage*: directory_missing: Type remaining existing endpoints
    
    Related to T645

commit 5d13cd7c545141a632e6e60e485cdc33e451f4e2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:07:34 2020 +0200

    storage*: skipped_content_missing: Type remaining existing endpoints
    
    Related to T645

commit 1a2aa70c687cbdac7655257ac3e6e572c8da43b4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:57:37 2020 +0200

    storage*: content_missing_per_sha1_git: Type remaining existing endpoints
    
    Related to T645

commit 15e48633a6a7beeccd233a38cdd456070fb4ae07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:55:39 2020 +0200

    storage*: content_missing_per_sha1: Type remaining existing endpoints
    
    Related to T645

commit b62afbbbdd95aaa91163c8907fa7237231d7c3a6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:13:43 2020 +0200

    storage*: content_missing: Unify and type remaining existing endpoints
    
    This updates the docstrings as well
    
    Related to T645

commit d6f26e45e9a62894ab2b888a15f8a4417c3b2ce0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:06:06 2020 +0200

    storage*: content_get_partition: Type remaining existing endpoints
    
    Related to T645

commit 864473370acdfd5acaf9afbfea9d16745b3c102f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:56:39 2020 +0200

    storage*: content_get_range: Type remaining existing endpoints
    
    Related to T645

commit c6da28289ec4f1be1eb1c1e78d561b8cc8dac937
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:45:31 2020 +0200

    storage*: content_get: Type remaining existing endpoints
    
    Related to T645

commit 25ebc48198d665cc3bb71c96a8ae54775ddf38a8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:42:12 2020 +0200

    storage*: content_update: Type remaining existing endpoints
    
    Related to T645

commit c32e224d23996705a6ca969696cc4715598ed796
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

commit 26ef01563f058bdfe77fcf5d2ca6a897e2894bc8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:24 2020 +0200

    storage*: check_config: Type remaining existing endpoints
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/689/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/689/console

Harbormaster failed remote builds in B14295: Diff 13052!Aug 5 2020, 11:01 AM
ardumont updated this revision to Diff 13055.EditedAug 5 2020, 11:12 AM

Build has FAILED

Exposed typed must match the interface so Optional is needed for the
revision_*log endpoints.

swh-public-ci added a comment.Aug 5 2020, 11:17 AM

Build is green

Patch application report for D3707 (id=13055)

Could not rebase; Attempt merge onto 26ef01563f...

Updating 26ef0156..760cbf6d
Fast-forward
 swh/storage/cassandra/converters.py |   2 +-
 swh/storage/cassandra/storage.py    | 110 ++++++++++++++++--------
 swh/storage/converters.py           |   6 +-
 swh/storage/filter.py               |  19 ++---
 swh/storage/in_memory.py            | 104 ++++++++++++++--------
 swh/storage/interface.py            | 166 ++++++++++++++++++++----------------
 swh/storage/storage.py              |  94 ++++++++++++--------
 7 files changed, 310 insertions(+), 191 deletions(-)
Changes applied before test
commit 760cbf6db540f9c22a19b9f1a76fc05ceeedb7df
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:33:49 2020 +0200

    storage*: revision_*log: Type remaining existing endpoints
    
    Related to T645

commit 38ee5255b0e0e34a31a7e154a9aac99c9ed16de9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:17:42 2020 +0200

    storage*: revision_get: Type remaining existing endpoints
    
    Related to T645

commit 8b6d18ef2f9e8834906229b16a367c00fb0b95b2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Aug 5 08:11:04 2020 +0200

    storage*: revision_missing: Type remaining existing endpoints
    
    Related to T645

commit 9f214bc745ee666071384b47da8a2f6cc7ab65da
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 23:04:26 2020 +0200

    storage*: directory_entry_get_by_path: Type remaining existing endpoints
    
    Related to T645

commit f9d09527a6ba682c7cdb487de3538fb58fa8b346
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:35:45 2020 +0200

    storage*: directory_ls: Type remaining existing endpoints
    
    Related to T645

commit fd5fd86b11a8dafeb564a3d07624becf57d90672
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:27:45 2020 +0200

    storage*: directory_missing: Type remaining existing endpoints
    
    Related to T645

commit 5d13cd7c545141a632e6e60e485cdc33e451f4e2
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 22:07:34 2020 +0200

    storage*: skipped_content_missing: Type remaining existing endpoints
    
    Related to T645

commit 1a2aa70c687cbdac7655257ac3e6e572c8da43b4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:57:37 2020 +0200

    storage*: content_missing_per_sha1_git: Type remaining existing endpoints
    
    Related to T645

commit 15e48633a6a7beeccd233a38cdd456070fb4ae07
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:55:39 2020 +0200

    storage*: content_missing_per_sha1: Type remaining existing endpoints
    
    Related to T645

commit b62afbbbdd95aaa91163c8907fa7237231d7c3a6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:13:43 2020 +0200

    storage*: content_missing: Unify and type remaining existing endpoints
    
    This updates the docstrings as well
    
    Related to T645

commit d6f26e45e9a62894ab2b888a15f8a4417c3b2ce0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 18:06:06 2020 +0200

    storage*: content_get_partition: Type remaining existing endpoints
    
    Related to T645

commit 864473370acdfd5acaf9afbfea9d16745b3c102f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:56:39 2020 +0200

    storage*: content_get_range: Type remaining existing endpoints
    
    Related to T645

commit c6da28289ec4f1be1eb1c1e78d561b8cc8dac937
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:45:31 2020 +0200

    storage*: content_get: Type remaining existing endpoints
    
    Related to T645

commit 25ebc48198d665cc3bb71c96a8ae54775ddf38a8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:42:12 2020 +0200

    storage*: content_update: Type remaining existing endpoints
    
    Related to T645

commit c32e224d23996705a6ca969696cc4715598ed796
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/691/ for more details.

Harbormaster completed remote builds in B14298: Diff 13055.Aug 5 2020, 11:17 AM
Closed by commit rDSTO8b6d18ef2f9e: storage*: revision_missing: Type remaining existing endpoints (authored by ardumont). · Explain WhyAug 5 2020, 11:55 AM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDSTO8b6d18ef2f9e: storage*: revision_missing: Type remaining existing endpoints.
ardumont added a commit: rDSTO38ee5255b0e0: storage*: revision_get: Type remaining existing endpoints.
ardumont added a commit: rDSTO760cbf6db540: storage*: revision_*log: Type remaining existing endpoints.
ardumont added a task: T2517: Add remaining missing types to swh.storage.interface.Aug 5 2020, 1:12 PM
ardumont removed a task: T645: Type swh-storage endpoints with swh.model objects.

Revision Contents
Changeset List

PathSize
swh/
storage/
cassandra/
2 lines
28 lines
6 lines
25 lines
31 lines
21 lines

Diff 13047

View Options

swh/storage/cassandra/converters.py

Loading...
View Options

swh/storage/cassandra/storage.py

Loading...
View Options

swh/storage/converters.py

Loading...
View Options

swh/storage/in_memory.py

Loading...
View Options

swh/storage/interface.py

Loading...
View Options

swh/storage/storage.py

Loading...
About · Developer information · Legal