Page MenuHomeSoftware Heritage

replayer: Fix isoformat datetime string for origin-visit
ClosedPublic

Authored by ardumont on Jun 25 2020, 11:22 AM.

Details

Summary

We no longer write datetime as strings in the journal. Still, the current
journal must have those old values within.

Related to D3336
Related to D3345

Depends on D3342

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
drop-origin-visit-fields
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13098
Build 19980: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 19979: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D3351 (id=11884)

Could not rebase; Attempt merge onto 621fc8d377...

Updating 621fc8d3..c6e6f339
Fast-forward
 requirements-swh.txt               |   2 +-
 sql/upgrades/155.sql               |  34 +++
 swh/storage/backfill.py            |  10 +-
 swh/storage/cassandra/cql.py       |   3 -
 swh/storage/cassandra/schema.py    |   3 -
 swh/storage/cassandra/storage.py   |  15 +-
 swh/storage/db.py                  |  27 ++-
 swh/storage/fixer.py               |  23 ++
 swh/storage/in_memory.py           |  18 +-
 swh/storage/sql/30-swh-schema.sql  |  11 +-
 swh/storage/sql/40-swh-func.sql    |   4 +-
 swh/storage/sql/60-swh-indexes.sql |   2 +-
 swh/storage/storage.py             |  10 +-
 swh/storage/tests/test_backfill.py |   5 +-
 swh/storage/tests/test_storage.py  | 423 ++++++++++++++++---------------------
 15 files changed, 284 insertions(+), 306 deletions(-)
 create mode 100644 sql/upgrades/155.sql
Changes applied before test
commit c6e6f33916aa5ac166de0eb4d80a81e01d4dad27
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jun 25 11:19:55 2020 +0200

    replayer: Fix isoformat datetime string for origin-visit
    
    We no longer write datetime as strings in the journal. Still, the current
    journal must have those old values within.
    
    Related to D3336
    Related to D3345

commit e5e80efeedecd90794e18abbf6815edb6b991a3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/349/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/349/console

Build is green

Patch application report for D3351 (id=11884)

Could not rebase; Attempt merge onto 621fc8d377...

Updating 621fc8d3..c6e6f339
Fast-forward
 requirements-swh.txt               |   2 +-
 sql/upgrades/155.sql               |  34 +++
 swh/storage/backfill.py            |  10 +-
 swh/storage/cassandra/cql.py       |   3 -
 swh/storage/cassandra/schema.py    |   3 -
 swh/storage/cassandra/storage.py   |  15 +-
 swh/storage/db.py                  |  27 ++-
 swh/storage/fixer.py               |  23 ++
 swh/storage/in_memory.py           |  18 +-
 swh/storage/sql/30-swh-schema.sql  |  11 +-
 swh/storage/sql/40-swh-func.sql    |   4 +-
 swh/storage/sql/60-swh-indexes.sql |   2 +-
 swh/storage/storage.py             |  10 +-
 swh/storage/tests/test_backfill.py |   5 +-
 swh/storage/tests/test_storage.py  | 423 ++++++++++++++++---------------------
 15 files changed, 284 insertions(+), 306 deletions(-)
 create mode 100644 sql/upgrades/155.sql
Changes applied before test
commit c6e6f33916aa5ac166de0eb4d80a81e01d4dad27
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jun 25 11:19:55 2020 +0200

    replayer: Fix isoformat datetime string for origin-visit
    
    We no longer write datetime as strings in the journal. Still, the current
    journal must have those old values within.
    
    Related to D3336
    Related to D3345

commit e5e80efeedecd90794e18abbf6815edb6b991a3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/350/ for more details.

This revision is now accepted and ready to land.Jun 25 2020, 1:00 PM