Page MenuHomeSoftware Heritage

origin-visit-update: Write visit-status objects to the journal
ClosedPublic

Authored by ardumont on Jun 8 2020, 4:28 PM.

Details

Summary

This makes origin_visit_update operations also write origin visit status to the
journal.

And reverts partial function call inlined in D3238 (as explained there)

Depends on D3238
Related to T2310

Depends D3241 (journal)

Test Plan

tox
tox (green on the backends part)


journal wise though, tox won't be happy. We need D3241 to land and be tagged first:

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont edited the test plan for this revision. (Show Details)

Build has FAILED

Patch application report for D3244 (id=11499)

Could not rebase; Attempt merge onto dcef916e5e...

Updating dcef916..b272f93
Fast-forward
 swh/storage/cassandra/storage.py  |  3 +-
 swh/storage/in_memory.py          | 67 ++++++++++++++++++--------------
 swh/storage/storage.py            | 13 +++----
 swh/storage/tests/test_storage.py | 80 ++++++++++++++++++++++++++++-----------
 4 files changed, 104 insertions(+), 59 deletions(-)
Changes applied before test
commit b272f93791022780129a4495a077a0dae01f654b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 8 16:27:11 2020 +0200

    origin-visit-update: Write visits to the journal
    
    Related to T2310

commit 3c0203cac19396d70f6770f6962d598c31ce69c0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 8 11:42:13 2020 +0200

    origin-visit-add: Write visit status to the journal
    
    This also makes the instruction order consistent across the different storage
    implementations. First, write objects to the journal, then write objects to the
    storage backend.
    
    Related to T2310

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/247/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/247/console

This revision is now accepted and ready to land.Jun 8 2020, 6:45 PM
ardumont edited the test plan for this revision. (Show Details)
  • Rebase on latest master
  • Fix journal related test to take into account origin-visit-status objects
  • Rework commit message
ardumont retitled this revision from origin-visit-update: Write visits to the journal to origin-visit-update: Write visit-status objects to the journal.Jun 9 2020, 2:35 PM

Build is green

Patch application report for D3244 (id=11518)

Could not rebase; Attempt merge onto 7eb44d412b...

Updating 7eb44d4..86d05fb
Fast-forward
 swh/storage/cassandra/storage.py       |  3 +-
 swh/storage/in_memory.py               | 67 ++++++++++++++++------------
 swh/storage/storage.py                 | 13 +++---
 swh/storage/tests/test_kafka_writer.py |  7 ++-
 swh/storage/tests/test_replay.py       |  2 +
 swh/storage/tests/test_storage.py      | 80 ++++++++++++++++++++++++----------
 6 files changed, 111 insertions(+), 61 deletions(-)
Changes applied before test
commit 86d05fb2751aeb8fd3cb7347ef788e04995cee45
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 8 16:27:11 2020 +0200

    origin-visit-update: Write visit status objects to the journal
    
    Related to T2310

commit 0860920774d1f907047443b4f0604ee6cbe2889b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 8 11:42:13 2020 +0200

    origin-visit-add: Write visit status to the journal
    
    This also makes the instruction order consistent across the different storage
    implementations. First, write objects to the journal, then write objects to the
    storage backend.
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/251/ for more details.