Page MenuHomeSoftware Heritage

tests: Relax checks on journal writes regarding origin-visit*/snapshot
ClosedPublic

Authored by ardumont on Jun 24 2020, 3:10 PM.

Details

Summary

According to a recent irc discussion about the too many insertions scenario.
The gist of it was, "unnecessary to check the multiple journal writes".

So this aligns the checks with other tests to ensure we find the expected
objects in the journal in a less exhaustive way (that means no strict equality).

This diff is about aligning according the discussion and avoiding conflating
D3342's work with this.

Depends on D3342

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build has FAILED

Patch application report for D3346 (id=11864)

Rebasing onto 621fc8d377...

Current branch diff-target is up to date.
Changes applied before test
commit 1683a12360653b9c58203f5eb9862ab7bfc7b54e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 15:06:00 2020 +0200

    Relax checks on journal writes regarding origin-visit*

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/343/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/343/console

It's a bit sad we cannot generically use sets for this (because some model objects are still unhashable), so fine for me.

This revision is now accepted and ready to land.Jun 24 2020, 5:07 PM

It's a bit sad we cannot generically use sets for this (because some model objects are still unhashable), so fine for me.

yeah, we'll eventually be able to rework this "soon" (hopefully).

Rebase on D3342 (so builds fine)

Depends on D3342

Build is green

Patch application report for D3346 (id=11885)

Could not rebase; Attempt merge onto 621fc8d377...

Updating 621fc8d3..fd240af6
Fast-forward
 requirements-swh.txt               |   2 +-
 sql/upgrades/155.sql               |  34 +++
 swh/storage/backfill.py            |  10 +-
 swh/storage/cassandra/cql.py       |   3 -
 swh/storage/cassandra/schema.py    |   3 -
 swh/storage/cassandra/storage.py   |  15 +-
 swh/storage/db.py                  |  27 ++-
 swh/storage/in_memory.py           |  18 +-
 swh/storage/sql/30-swh-schema.sql  |  11 +-
 swh/storage/sql/40-swh-func.sql    |   4 +-
 swh/storage/sql/60-swh-indexes.sql |   2 +-
 swh/storage/storage.py             |  10 +-
 swh/storage/tests/test_backfill.py |   5 +-
 swh/storage/tests/test_storage.py  | 432 ++++++++++++++++---------------------
 14 files changed, 267 insertions(+), 309 deletions(-)
 create mode 100644 sql/upgrades/155.sql
Changes applied before test
commit fd240af6234e205b4593d4a3691fd9c01cd71ac9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 15:06:00 2020 +0200

    Relax checks on journal writes regarding origin-visit*

commit e5e80efeedecd90794e18abbf6815edb6b991a3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/351/ for more details.

Build is green

Patch application report for D3346 (id=11894)

Rebasing onto c6e6f33916...

Current branch diff-target is up to date.
Changes applied before test
commit 12d729b9cf3aaeb7765ca513282469f762faa243
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 15:06:00 2020 +0200

    Relax checks on journal writes regarding origin-visit*

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/353/ for more details.