Page MenuHomeSoftware Heritage
Differential D3328

Deprecate the origin_add_one() endpoint
ClosedPublic
Actions

Authored by douardda on Jun 22 2020, 11:38 AM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Commits
rDSTO621fc8d37795: Deprecate the origin_add_one() endpoint
Summary

This endpoint is not really useful since the origin_add() can be used
instead.

Using a single API endpoint would also make the API a bit more consistant
(most other endpoints only provide a xxx_add endpoint) ; having a single
endpoint per object_type make is enough and make the whole API simpler.

Depends on D3327.

Diff Detail

Unit TestsFailed
View All

TimeTest
663 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.tests.test_storage.TestStorage::test_origin_visit_get_latest_filter_type
swh_storage_backend_config = {'cls': 'local', 'db': 'postgresql://postgres@127.0.0.1:26504/tests', 'journal_writer': {'cls': 'memory'}, 'objstorage': {'args': {}, 'cls': 'memory'}} @pytest.fixture
0 msCoverage Data
4 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_content
4 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_content
2 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.storage.fixer::swh.storage.fixer._fix_origin
View Full Test Results (1 Failed · 1,496 Passed · 34 Skipped)

Event Timeline

douardda created this revision.Jun 22 2020, 11:38 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptJun 22 2020, 11:38 AM
swh-public-ci added a comment.Jun 22 2020, 11:52 AM

Build has FAILED

Patch application report for D3328 (id=11787)

Could not rebase; Attempt merge onto 041543df09...

Merge made by the 'recursive' strategy.
 mypy.ini                               |  3 ++
 requirements.txt                       |  1 +
 swh/storage/cassandra/storage.py       | 14 ++++---
 swh/storage/in_memory.py               | 15 +++++--
 swh/storage/interface.py               |  7 +++-
 swh/storage/storage.py                 | 38 +++++++++--------
 swh/storage/tests/algos/test_origin.py |  7 +++-
 swh/storage/tests/test_storage.py      | 76 ++++++++++++++++++----------------
 swh/storage/validate.py                |  7 +++-
 9 files changed, 102 insertions(+), 66 deletions(-)
Changes applied before test
commit a11d8545546562aa51f15643965eda229ba9a9bc
Merge: 041543d b2aa7b5
Author: Jenkins user <jenkins@localhost>
Date:   Mon Jun 22 09:45:19 2020 +0000

    Merge branch 'diff-target' into HEAD

commit b2aa7b5f5081d9e7c33f90c3ba693f5e59ad3624
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 22 11:27:54 2020 +0200

    Deprecate the origin_add_one() endpoint
    
    This endpoint is not really useful since the origin_add() can be used
    instead.
    
    Using a single API endpoint would also make the API a bit more consistant
    (most other endpoints only provide a xxx_add endpoint) ; having a single
    endpoint per object_type make is enough and make the whole API simpler.

commit e937b56c573afad2c207324d769ff42ed6f29cab
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 18 18:28:51 2020 +0200

    Refactor the Storage.add_origin API
    
    make it consistent with other add_xxx methods, ie. make it return a
    summary dict.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/324/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/324/console

Harbormaster failed remote builds in B12999: Diff 11787!Jun 22 2020, 11:52 AM
ardumont accepted this revision.Jun 22 2020, 12:34 PM
This revision is now accepted and ready to land.Jun 22 2020, 12:34 PM
douardda updated this revision to Diff 11797.Jun 22 2020, 2:39 PM

rebase

swh-public-ci added a comment.Jun 22 2020, 3:00 PM

Build has FAILED

Patch application report for D3328 (id=11797)

Could not rebase; Attempt merge onto 041543df09...

Merge made by the 'recursive' strategy.
 mypy.ini                               |  3 ++
 requirements.txt                       |  1 +
 swh/storage/cassandra/storage.py       | 18 +++++---
 swh/storage/in_memory.py               | 15 +++++--
 swh/storage/interface.py               |  7 ++-
 swh/storage/storage.py                 | 38 +++++++++-------
 swh/storage/tests/algos/test_origin.py |  7 ++-
 swh/storage/tests/test_storage.py      | 82 +++++++++++++++++++---------------
 swh/storage/validate.py                |  7 ++-
 9 files changed, 112 insertions(+), 66 deletions(-)
Changes applied before test
commit 90929250be7d3a733685cbbb4ff4f7ead8537e74
Merge: 041543d 5737ce2
Author: Jenkins user <jenkins@localhost>
Date:   Mon Jun 22 12:53:29 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 5737ce2d607b84a3443a8b1b6d17d7157f4169d7
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 22 11:27:54 2020 +0200

    Deprecate the origin_add_one() endpoint
    
    This endpoint is not really useful since the origin_add() can be used
    instead.
    
    Using a single API endpoint would also make the API a bit more consistant
    (most other endpoints only provide a xxx_add endpoint) ; having a single
    endpoint per object_type make is enough and make the whole API simpler.

commit b1251351432289622b50ebb05ae394d069022fb3
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 18 18:28:51 2020 +0200

    Refactor the Storage.add_origin API
    
    make it consistent with other add_xxx methods, ie. make it return a
    summary dict.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/328/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/328/console

Harbormaster failed remote builds in B13009: Diff 11797!Jun 22 2020, 3:00 PM
douardda updated this revision to Diff 11814.Jun 22 2020, 5:05 PM

rebase

swh-public-ci added a comment.Jun 22 2020, 5:19 PM

Build has FAILED

Patch application report for D3328 (id=11814)

Could not rebase; Attempt merge onto 2d497ff07c...

Auto-merging swh/storage/tests/algos/test_origin.py
Merge made by the 'recursive' strategy.
 mypy.ini                               |  3 ++
 requirements.txt                       |  1 +
 swh/storage/cassandra/storage.py       | 17 +++++--
 swh/storage/in_memory.py               | 15 ++++--
 swh/storage/interface.py               |  7 ++-
 swh/storage/storage.py                 | 38 +++++++++-------
 swh/storage/tests/algos/test_origin.py |  7 ++-
 swh/storage/tests/test_storage.py      | 83 +++++++++++++++++++---------------
 swh/storage/validate.py                |  7 ++-
 9 files changed, 111 insertions(+), 67 deletions(-)
Changes applied before test
commit c5bde4db17faabc38281d6959f038ad60ba94891
Merge: 2d497ff0 4f156595
Author: Jenkins user <jenkins@localhost>
Date:   Mon Jun 22 15:12:32 2020 +0000

    Merge branch 'diff-target' into HEAD

commit 4f156595f7894a6e346fed1e5f5f3262ea3ffbab
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 22 11:27:54 2020 +0200

    Deprecate the origin_add_one() endpoint
    
    This endpoint is not really useful since the origin_add() can be used
    instead.
    
    Using a single API endpoint would also make the API a bit more consistant
    (most other endpoints only provide a xxx_add endpoint) ; having a single
    endpoint per object_type make is enough and make the whole API simpler.

commit f6aef1c74cb30d9e890ee538baf1010157542967
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 18 18:28:51 2020 +0200

    Make Storage.add_origin() return a sumary dict
    
    make it consistent with other add_xxx methods by making it return a
    summary dict `{"origin:add": int}`.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/332/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/332/console

Harbormaster failed remote builds in B13029: Diff 11814!Jun 22 2020, 5:19 PM
douardda updated this revision to Diff 11837.Jun 23 2020, 2:55 PM

rebase

swh-public-ci added a comment.Jun 23 2020, 3:08 PM

Build has FAILED

Patch application report for D3328 (id=11837)

Could not rebase; Attempt merge onto 2d497ff07c...

Auto-merging swh/storage/tests/algos/test_origin.py
Merge made by the 'recursive' strategy.
 mypy.ini                               |  3 ++
 requirements.txt                       |  1 +
 swh/storage/cassandra/storage.py       | 17 +++++--
 swh/storage/in_memory.py               | 15 ++++--
 swh/storage/interface.py               |  7 ++-
 swh/storage/storage.py                 | 38 ++++++++-------
 swh/storage/tests/algos/test_origin.py |  7 ++-
 swh/storage/tests/test_storage.py      | 87 +++++++++++++++++++---------------
 swh/storage/validate.py                |  7 ++-
 9 files changed, 112 insertions(+), 70 deletions(-)
Changes applied before test
commit 2dd82ef71950366b244b21ecce597d5cf7e51166
Merge: 2d497ff0 d229f4ff
Author: Jenkins user <jenkins@localhost>
Date:   Tue Jun 23 13:02:24 2020 +0000

    Merge branch 'diff-target' into HEAD

commit d229f4ff92bb55fab9b75e0c9f6292721f536438
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 22 11:27:54 2020 +0200

    Deprecate the origin_add_one() endpoint
    
    This endpoint is not really useful since the origin_add() can be used
    instead.
    
    Using a single API endpoint would also make the API a bit more consistant
    (most other endpoints only provide a xxx_add endpoint) ; having a single
    endpoint per object_type make is enough and make the whole API simpler.

commit ae6f4772a0bf8e4003479a87844563f515cd6eba
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 18 18:28:51 2020 +0200

    Make Storage.add_origin() return a sumary dict
    
    make it consistent with other add_xxx methods by making it return a
    summary dict `{"origin:add": int}`.

commit 9f9abcdd577cc00cfb937cfc0bbc4878d985b6e6
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 22 17:47:26 2020 +0200

    Simplify/fix test_origin_visit_status_add_twice
    
    do not make assumptions on how the backend storage will manage idempotence
    related to the journal.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/336/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/336/console

Harbormaster failed remote builds in B13052: Diff 11837!Jun 23 2020, 3:08 PM
douardda updated this revision to Diff 11841.Jun 23 2020, 4:08 PM

rebase

swh-public-ci added a comment.Jun 23 2020, 4:21 PM

Build has FAILED

Patch application report for D3328 (id=11841)

Could not rebase; Attempt merge onto 2d497ff07c...

Updating 2d497ff0..621fc8d3
Fast-forward
 mypy.ini                               |  3 ++
 requirements.txt                       |  1 +
 swh/storage/cassandra/storage.py       | 19 +++++---
 swh/storage/in_memory.py               | 15 ++++--
 swh/storage/interface.py               |  7 ++-
 swh/storage/storage.py                 | 38 +++++++++-------
 swh/storage/tests/algos/test_origin.py |  7 ++-
 swh/storage/tests/test_storage.py      | 83 +++++++++++++++++++---------------
 swh/storage/validate.py                |  7 ++-
 9 files changed, 112 insertions(+), 68 deletions(-)
Changes applied before test
commit 621fc8d3779535656e15499464c44acecd7eb8bf
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 22 11:27:54 2020 +0200

    Deprecate the origin_add_one() endpoint
    
    This endpoint is not really useful since the origin_add() can be used
    instead.
    
    Using a single API endpoint would also make the API a bit more consistant
    (most other endpoints only provide a xxx_add endpoint) ; having a single
    endpoint per object_type make is enough and make the whole API simpler.

commit fb603e1b9874a3bbb393f7653ec5004bfa18b251
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 18 18:28:51 2020 +0200

    Make Storage.add_origin() return a sumary dict
    
    make it consistent with other add_xxx methods by making it return a
    summary dict `{"origin:add": int}`.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/338/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/338/console

Harbormaster failed remote builds in B13056: Diff 11841!Jun 23 2020, 4:21 PM
swh-public-ci added a comment.Jun 23 2020, 4:44 PM

Build is green

Patch application report for D3328 (id=11841)

Could not rebase; Attempt merge onto 2d497ff07c...

Updating 2d497ff0..621fc8d3
Fast-forward
 mypy.ini                               |  3 ++
 requirements.txt                       |  1 +
 swh/storage/cassandra/storage.py       | 19 +++++---
 swh/storage/in_memory.py               | 15 ++++--
 swh/storage/interface.py               |  7 ++-
 swh/storage/storage.py                 | 38 +++++++++-------
 swh/storage/tests/algos/test_origin.py |  7 ++-
 swh/storage/tests/test_storage.py      | 83 +++++++++++++++++++---------------
 swh/storage/validate.py                |  7 ++-
 9 files changed, 112 insertions(+), 68 deletions(-)
Changes applied before test
commit 621fc8d3779535656e15499464c44acecd7eb8bf
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 22 11:27:54 2020 +0200

    Deprecate the origin_add_one() endpoint
    
    This endpoint is not really useful since the origin_add() can be used
    instead.
    
    Using a single API endpoint would also make the API a bit more consistant
    (most other endpoints only provide a xxx_add endpoint) ; having a single
    endpoint per object_type make is enough and make the whole API simpler.

commit fb603e1b9874a3bbb393f7653ec5004bfa18b251
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 18 18:28:51 2020 +0200

    Make Storage.add_origin() return a sumary dict
    
    make it consistent with other add_xxx methods by making it return a
    summary dict `{"origin:add": int}`.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/340/ for more details.

This revision was landed with ongoing or failed builds.Jun 23 2020, 4:49 PM
Closed by commit rDSTO621fc8d37795: Deprecate the origin_add_one() endpoint (authored by douardda). · Explain Why
This revision was automatically updated to reflect the committed changes.
douardda added a commit: rDSTO621fc8d37795: Deprecate the origin_add_one() endpoint.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
3 lines--
1 line--
swh/
storage/
cassandra/
2 lines93%
Loading...
3 lines96%
Loading...
1 line65%
Loading...
2 lines92%
Loading...
3 lines98%
Loading...

Diff 11841

View Options

mypy.ini

Loading...
View Options

requirements.txt

Loading...
View Options

swh/storage/cassandra/storage.py

Loading...
View Options

swh/storage/in_memory.py

Loading...
View Options

swh/storage/interface.py

Loading...
View Options

swh/storage/storage.py

Loading...
View Options

swh/storage/validate.py

Loading...
About · Developer information · Legal