Page MenuHomeSoftware Heritage

Add persistence of the field OriginVisitStatus.type
ClosedPublic

Authored by vsellier on Jan 13 2021, 4:47 PM.

Details

Summary

(!) A new database upgrade is needed (165.sql) for postgresql backend

Related to T2964
Depends on D4848
Depends on D4857

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 18393
Build 28422: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 28421: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D4858 (id=17190)

Could not rebase; Attempt merge onto 728c3eeaae...

Updating 728c3eea..031ef036
Fast-forward
 requirements-swh-journal.txt             |  2 +-
 requirements-swh.txt                     |  2 +-
 sql/upgrades/165.sql                     | 12 +++++++
 swh/storage/cassandra/model.py           |  7 +++-
 swh/storage/cassandra/schema.py          |  1 +
 swh/storage/cassandra/storage.py         | 12 +++++++
 swh/storage/postgresql/db.py             |  2 ++
 swh/storage/postgresql/storage.py        | 18 +++++++++-
 swh/storage/sql/30-schema.sql            |  2 ++
 swh/storage/tests/algos/test_origin.py   | 18 ++++++----
 swh/storage/tests/algos/test_snapshot.py | 15 +++++---
 swh/storage/tests/storage_tests.py       | 60 ++++++++++++++++++++++----------
 12 files changed, 117 insertions(+), 34 deletions(-)
 create mode 100644 sql/upgrades/165.sql
Changes applied before test
commit 031ef03629a0f41cd26ea61be2cf4b8129e508dd
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Wed Jan 13 16:34:08 2021 +0100

    Add persistence of the field OriginVisitStatus.type
    
    (!) A new database upgrade is needed (165.sql) for postgresql backend
    
    Related to T2964

commit 0b44b37254974db02f87339448f4629fa9a91ded
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Tue Jan 12 18:26:04 2021 +0100

    Adapt cassandra storage to ignore the new OriginVisitStatus.type field
    
    Depends on D4848
    Related to T2443

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1098/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1098/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 4:48 PM
Harbormaster failed remote builds in B18334: Diff 17190!

Build is green

Patch application report for D4858 (id=17190)

Rebasing onto 0b44b37254...

Current branch diff-target is up to date.
Changes applied before test
commit 031ef03629a0f41cd26ea61be2cf4b8129e508dd
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Wed Jan 13 16:34:08 2021 +0100

    Add persistence of the field OriginVisitStatus.type
    
    (!) A new database upgrade is needed (165.sql) for postgresql backend
    
    Related to T2964

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1100/ for more details.

vlorentz added inline comments.
swh/storage/cassandra/model.py
194
tenma added a subscriber: tenma.

TGTM, but needs a look by someone more knowledgeable about the DBs

This revision is now accepted and ready to land.Jan 15 2021, 11:43 AM
tenma requested changes to this revision.Jan 15 2021, 11:45 AM

did not see the requested change on clustering keys by vlorentz

This revision now requires changes to proceed.Jan 15 2021, 11:45 AM

Remove type from the clustering key of OriginVisitStatus

Build is green

Patch application report for D4858 (id=17250)

Rebasing onto da55308f3d...

Current branch diff-target is up to date.
Changes applied before test
commit 042f23cb0c5cc1e0481ee7788c806b5e5851cebc
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Wed Jan 13 16:34:08 2021 +0100

    Add persistence of the field OriginVisitStatus.type
    
    (!) A new database upgrade is needed (165.sql) for postgresql backend
    
    Related to T2964

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1102/ for more details.

Build is green

Patch application report for D4858 (id=17251)

Rebasing onto da55308f3d...

Current branch diff-target is up to date.
Changes applied before test
commit c24d35f86a0667f67f4daf4d3eefa0272bf981ef
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Wed Jan 13 16:34:08 2021 +0100

    Add persistence of the field OriginVisitStatus.type
    
    (!) A new database upgrade is needed (165.sql) for postgresql backend
    
    Related to T2964

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1103/ for more details.

This revision was not accepted when it landed; it landed in state Needs Review.Jan 15 2021, 1:58 PM
This revision was automatically updated to reflect the committed changes.