Paths

Table of Contentst

Differential D3698

storage*: origin_get_by_sha1: Type remaining existing endpoints
ClosedPublic
Actions

Authored by ardumont on Aug 4 2020, 6:02 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T2517: Add remaining missing types to swh.storage.interface

Commits

rDSTOc32e224d2399: storage*: origin_get_by_sha1: Type remaining existing endpoints

Summary

Last origin endpoint. I'll change it soon to return origin objects.
But let's type remaining as is for now.

Related to T645

Test Plan

tox

Diff Detail

Repository

rDSTO Storage manager

Branch

master

Lint

Lint Skipped

Unit

Unit Tests Skipped

Build Status

Buildable 14272
Build 21945: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 21944: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	308 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_cassandra.TestCassandraStorage::test_types self = <swh.storage.tests.test_cassandra.TestCassandraStorage object at 0x7f7710144eb8> swh_storage_backend_config = {'cls': 'cassandra', 'hosts': ['127.0.0.1'], 'journal_writer': {'cls': 'memory'}, 'keyspace': 'db77e4b04ab0f0136d49', ...}
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_content
	1 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_origin
	2 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_origin_visit
	2 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_revision
		View Full Test Results (1 Failed · 731 Passed · 15 Skipped)

Event Timeline

ardumont created this revision.Aug 4 2020, 6:02 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptAug 4 2020, 6:02 PM

Build has FAILED

Patch application report for D3698 (id=13029)

Could not rebase; Attempt merge onto a5232b7a7b...

Updating a5232b7a..8d740295
Fast-forward
 swh/storage/cassandra/storage.py |  2 +-
 swh/storage/in_memory.py         |  9 ++++++---
 swh/storage/interface.py         | 14 +++++++-------
 swh/storage/storage.py           |  6 ++++--
 4 files changed, 18 insertions(+), 13 deletions(-)

Changes applied before test

commit 8d7402954e16a8b1e2a3b3f8b6bd0547ba3995e6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

commit 26ef01563f058bdfe77fcf5d2ca6a897e2894bc8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:24 2020 +0200

    storage*: check_config: Type remaining existing endpoints
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/666/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/666/console

Harbormaster failed remote builds in B14272: Diff 13029!Aug 4 2020, 6:10 PM

ardumont added inline comments.Aug 4 2020, 6:19 PM

swh/storage/in_memory.py
677	ardu: `-> List` !!!

ardumont added inline comments.Aug 4 2020, 6:34 PM

swh/storage/in_memory.py
677	ah yeah, not now, it's yielding here.

Fix

Build is green

Patch application report for D3698 (id=13030)

Could not rebase; Attempt merge onto a5232b7a7b...

Updating a5232b7a..c32e224d
Fast-forward
 swh/storage/cassandra/storage.py | 12 ++++++------
 swh/storage/in_memory.py         |  9 ++++++---
 swh/storage/interface.py         | 14 +++++++-------
 swh/storage/storage.py           |  6 ++++--
 4 files changed, 23 insertions(+), 18 deletions(-)

Changes applied before test

commit c32e224d23996705a6ca969696cc4715598ed796
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:46 2020 +0200

    storage*: origin_get_by_sha1: Type remaining existing endpoints
    
    Related to T645

commit 26ef01563f058bdfe77fcf5d2ca6a897e2894bc8
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Aug 4 17:59:24 2020 +0200

    storage*: check_config: Type remaining existing endpoints
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/667/ for more details.

Harbormaster completed remote builds in B14273: Diff 13030.Aug 4 2020, 6:44 PM

vlorentz added a subscriber: vlorentz.Aug 5 2020, 8:53 AM

vlorentz added inline comments.

swh/storage/in_memory.py
677–678	why this change? We should move toward returning lists, so that's a step backward

ardumont added inline comments.Aug 5 2020, 8:57 AM

swh/storage/in_memory.py
677–678	well, i'm aligning with what the production does (pg). so we have signature that matches. so that's a step backward no, that's not running anywhere besides test. I can probably change towards what you say though. Make the pg implementation returns a list as is and change the type with a list.

vlorentz added inline comments.Aug 5 2020, 8:58 AM

swh/storage/in_memory.py
677–678	so we have signature that matches. The signature also matches if you return a list, because lists are iterables.

To avoid too much hassle with git/arc/jenkins (this is a base root of my other changes/diff you accepted...), i'll commit this as is and improve as you suggest in another diff if you don't mind [1]

[1] D3708

ardumont mentioned this in D3708: storage*: origin_get_by_sha1: Drop generator from pgstorage.Aug 5 2020, 9:55 AM

vlorentz accepted this revision.Aug 5 2020, 10:22 AM

This revision is now accepted and ready to land.Aug 5 2020, 10:22 AM

Closed by commit rDSTOc32e224d2399: storage*: origin_get_by_sha1: Type remaining existing endpoints (authored by ardumont). · Explain WhyAug 5 2020, 11:17 AM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDSTOc32e224d2399: storage*: origin_get_by_sha1: Type remaining existing endpoints.

ardumont added a task: T2517: Add remaining missing types to swh.storage.interface.Aug 5 2020, 1:12 PM

ardumont removed a task: T645: Type swh-storage endpoints with swh.model objects.