Page MenuHomeSoftware Heritage

storage*: Drop obsolete fields from origin_visit
ClosedPublic

Authored by ardumont on Jun 24 2020, 11:00 AM.

Details

Summary

This removes:

  • removes those columns from the backends (cass, pg)
  • removes the status, metadata and snapshot from OriginVisit object instantiation
  • creates the initial origin-visit-status status to "created" when adding a new origin-visit

Depends on journal (test data update D3344, code adaptation D3345).

This also fixes the current failed build master [1]

[1] https://jenkins.softwareheritage.org/job/DSTO/job/tests/1299/console

Related to T2310

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
drop-origin-visit-fields
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13095
Build 19974: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 19973: arc lint + arc unit

Event Timeline

ardumont edited the summary of this revision. (Show Details)

Build was aborted

Patch application report for D3342 (id=11853)

Rebasing onto 621fc8d377...

Current branch diff-target is up to date.
Changes applied before test
commit 4787c8c332254547ef541cfea22589143bb94d5d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/341/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/341/console

Update backfiller case

Depends on D3344 for the tests (journal test data) to pass though

Build has FAILED

Patch application report for D3342 (id=11861)

Rebasing onto 621fc8d377...

Current branch diff-target is up to date.
Changes applied before test
commit 9a5cc1d0f95707a00217fb60f469fe6a91620cb1
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/342/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/342/console

Build has FAILED

Patch application report for D3342 (id=11861)

Rebasing onto 621fc8d377...

Current branch diff-target is up to date.
Changes applied before test
commit 9a5cc1d0f95707a00217fb60f469fe6a91620cb1
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/347/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/347/console

This comment was removed by ardumont.

ardumont removed comment

This comment was not meant for that diff

planned changes

was not meant for that diff either... but it needs some rebase anyway and some
commented dead code to remove. So taking the opportunity here ;)

swh/storage/tests/test_storage.py
2247

at it was initially the same date as the origin-visit-status from the origin-add call.
This one got dropped silently on conflict (origin, visit, date) ignore style.

Drop unnecessary commented code

Build is green

Patch application report for D3342 (id=11881)

Rebasing onto 621fc8d377...

Current branch diff-target is up to date.
Changes applied before test
commit e5e80efeedecd90794e18abbf6815edb6b991a3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/348/ for more details.

vlorentz added inline comments.
swh/storage/sql/60-swh-indexes.sql
129

could you give it a name?

(to make sure we don't get different names on prod vs dev)

This revision is now accepted and ready to land.Jun 25 2020, 12:58 PM
swh/storage/sql/60-swh-indexes.sql
129

that's a good point ;)

will do.