Page MenuHomeSoftware Heritage

replayer: Fix isoformat datetime string for origin-visit
ClosedPublic

Authored by ardumont on Thu, Jun 25, 11:22 AM.

Details

Summary

We no longer write datetime as strings in the journal. Still, the current
journal must have those old values within.

Related to D3336
Related to D3345

Depends on D3342

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Thu, Jun 25, 11:22 AM

Build has FAILED

Patch application report for D3351 (id=11884)

Could not rebase; Attempt merge onto 621fc8d377...

Updating 621fc8d3..c6e6f339
Fast-forward
 requirements-swh.txt               |   2 +-
 sql/upgrades/155.sql               |  34 +++
 swh/storage/backfill.py            |  10 +-
 swh/storage/cassandra/cql.py       |   3 -
 swh/storage/cassandra/schema.py    |   3 -
 swh/storage/cassandra/storage.py   |  15 +-
 swh/storage/db.py                  |  27 ++-
 swh/storage/fixer.py               |  23 ++
 swh/storage/in_memory.py           |  18 +-
 swh/storage/sql/30-swh-schema.sql  |  11 +-
 swh/storage/sql/40-swh-func.sql    |   4 +-
 swh/storage/sql/60-swh-indexes.sql |   2 +-
 swh/storage/storage.py             |  10 +-
 swh/storage/tests/test_backfill.py |   5 +-
 swh/storage/tests/test_storage.py  | 423 ++++++++++++++++---------------------
 15 files changed, 284 insertions(+), 306 deletions(-)
 create mode 100644 sql/upgrades/155.sql
Changes applied before test
commit c6e6f33916aa5ac166de0eb4d80a81e01d4dad27
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jun 25 11:19:55 2020 +0200

    replayer: Fix isoformat datetime string for origin-visit
    
    We no longer write datetime as strings in the journal. Still, the current
    journal must have those old values within.
    
    Related to D3336
    Related to D3345

commit e5e80efeedecd90794e18abbf6815edb6b991a3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/349/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/349/console

Build is green

Patch application report for D3351 (id=11884)

Could not rebase; Attempt merge onto 621fc8d377...

Updating 621fc8d3..c6e6f339
Fast-forward
 requirements-swh.txt               |   2 +-
 sql/upgrades/155.sql               |  34 +++
 swh/storage/backfill.py            |  10 +-
 swh/storage/cassandra/cql.py       |   3 -
 swh/storage/cassandra/schema.py    |   3 -
 swh/storage/cassandra/storage.py   |  15 +-
 swh/storage/db.py                  |  27 ++-
 swh/storage/fixer.py               |  23 ++
 swh/storage/in_memory.py           |  18 +-
 swh/storage/sql/30-swh-schema.sql  |  11 +-
 swh/storage/sql/40-swh-func.sql    |   4 +-
 swh/storage/sql/60-swh-indexes.sql |   2 +-
 swh/storage/storage.py             |  10 +-
 swh/storage/tests/test_backfill.py |   5 +-
 swh/storage/tests/test_storage.py  | 423 ++++++++++++++++---------------------
 15 files changed, 284 insertions(+), 306 deletions(-)
 create mode 100644 sql/upgrades/155.sql
Changes applied before test
commit c6e6f33916aa5ac166de0eb4d80a81e01d4dad27
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jun 25 11:19:55 2020 +0200

    replayer: Fix isoformat datetime string for origin-visit
    
    We no longer write datetime as strings in the journal. Still, the current
    journal must have those old values within.
    
    Related to D3336
    Related to D3345

commit e5e80efeedecd90794e18abbf6815edb6b991a3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/350/ for more details.

vlorentz accepted this revision.Thu, Jun 25, 1:00 PM
This revision is now accepted and ready to land.Thu, Jun 25, 1:00 PM