Differential D3651

storage*: Type origin_search(...) -> PagedResult[Origin]
ClosedPublic
Actions

Authored by ardumont on Jul 30 2020, 4:10 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T645: Type swh-storage endpoints with swh.model objects

Commits

rDSTOcf9f44e80532: storage*: Type origin_search(...) -> PagedResult[Origin]

Summary

This unifies the how the pagination is now done, abstracting away how the
pagination is dealt with internally.

This does not touch the internal logic of limit/offset used within.

Impacts swh-web:

grep -r 'origin_search(' ../*/swh/** | grep -v "swh-storage" | grep -v "swh-search"
../swh-web/swh/web/api/views/origin.py:def api_origin_search(request, url_pattern):
../swh-web/swh/web/common/service.py:        results = search.origin_search(
../swh-web/swh/web/common/service.py:        origins_raw = storage.origin_search(
../swh-web/swh/web/assets/src/bundles/browse/origin-search.js:    baseSearchUrl = new URL(Urls.api_1_origin_search(searchQueryText), window.location);
../swh-web/swh/web/tests/api/views/test_origin.py:def test_api_origin_search(api_client, mocker, backend):

Related to T645

Test Plan

tox

Diff Detail

Repository

rDSTO Storage manager

Branch

open-snapshot-id

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 14096
Build 21648: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 21647: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	1,590 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_api_client.TestStorage::test_types self = <swh.storage.tests.test_api_client.TestStorage object at 0x7fc6374397f0> swh_storage_backend_config = {'cls': 'local', 'db': 'postgresql://postgres@127.0.0.1:29649/tests', 'journal_writer': {'cls': 'memory'}, 'objstorage': {'args': {}, 'cls': 'memory'}}
	54 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_api_client.TestStorageGeneratedData::test_origin_count self = <swh.storage.tests.test_storage.TestStorageGeneratedData object at 0x7fc6371ef198> swh_storage = <RemoteStorage url=mock://example.com/> sample_data = <swh.storage.tests.storage_data.StorageData object at 0x7fc6374bdbe0>
	61 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_api_client.TestStorageGeneratedData::test_origin_count_with_visit_no_visits self = <swh.storage.tests.test_storage.TestStorageGeneratedData object at 0x7fc6374bdf98> swh_storage = <RemoteStorage url=mock://example.com/> sample_data = <swh.storage.tests.storage_data.StorageData object at 0x7fc6372d95f8>
	68 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_api_client.TestStorageGeneratedData::test_origin_count_with_visit_with_visits_and_snapshot self = <swh.storage.tests.test_storage.TestStorageGeneratedData object at 0x7fc6378f9eb8> swh_storage = <RemoteStorage url=mock://example.com/> sample_data = <swh.storage.tests.storage_data.StorageData object at 0x7fc6378f9208>
	67 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_api_client.TestStorageGeneratedData::test_origin_count_with_visit_with_visits_no_snapshot self = <swh.storage.tests.test_storage.TestStorageGeneratedData object at 0x7fc6372ee748> swh_storage = <RemoteStorage url=mock://example.com/> sample_data = <swh.storage.tests.storage_data.StorageData object at 0x7fc6372eef28>
		View Full Test Results (10 Failed · 743 Passed · 17 Skipped)

Event Timeline

ardumont created this revision.Jul 30 2020, 4:10 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 30 2020, 4:10 PM

vlorentz accepted this revision.Jul 30 2020, 4:14 PM

This revision is now accepted and ready to land.Jul 30 2020, 4:14 PM

Build has FAILED

Patch application report for D3651 (id=12859)

Rebasing onto 4d52fc1d07...

Current branch diff-target is up to date.

Changes applied before test

commit 9bb6868cf4a6afd99d8944acf79c135c4195f79b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 30 16:07:53 2020 +0200

    storage*: Type origin_search(...) -> PagedResult[Origin]
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/626/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/626/console

Harbormaster failed remote builds in B14096: Diff 12859!Jul 30 2020, 4:17 PM

Fix missing type annotation on storage implementation
Fix in-memory origin-count implementation which reuses origin-search internally

Build was aborted

Patch application report for D3651 (id=12860)

Rebasing onto 4d52fc1d07...

Current branch diff-target is up to date.

Changes applied before test

commit cf9f44e80532c97c11a5cd00bd2f85e735d84998
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 30 16:07:53 2020 +0200

    storage*: Type origin_search(...) -> PagedResult[Origin]
    
    Related to T645

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/627/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/627/console

Harbormaster failed remote builds in B14097: Diff 12860!Jul 30 2020, 4:43 PM

Build is green

Patch application report for D3651 (id=12860)

Rebasing onto 4d52fc1d07...

Current branch diff-target is up to date.

Changes applied before test

commit cf9f44e80532c97c11a5cd00bd2f85e735d84998
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 30 16:07:53 2020 +0200

    storage*: Type origin_search(...) -> PagedResult[Origin]
    
    Related to T645

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/628/ for more details.

Harbormaster completed remote builds in B14097: Diff 12860.Jul 30 2020, 4:52 PM

Closed by commit rDSTOcf9f44e80532: storage*: Type origin_search(...) -> PagedResult[Origin] (authored by ardumont). · Explain WhyJul 30 2020, 5:02 PM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDSTOcf9f44e80532: storage*: Type origin_search(...) -> PagedResult[Origin].