Page MenuHomeSoftware Heritage

tests: Relax checks on journal writes regarding origin-visit*/snapshot
ClosedPublic

Authored by ardumont on Jun 24 2020, 3:10 PM.

Details

Summary

According to a recent irc discussion about the too many insertions scenario.
The gist of it was, "unnecessary to check the multiple journal writes".

So this aligns the checks with other tests to ensure we find the expected
objects in the journal in a less exhaustive way (that means no strict equality).

This diff is about aligning according the discussion and avoiding conflating
D3342's work with this.

Depends on D3342

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
relax-checks
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13099
Build 19982: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 19981: arc lint + arc unit

Event Timeline

ardumont created this revision.Jun 24 2020, 3:10 PM

Build has FAILED

Patch application report for D3346 (id=11864)

Rebasing onto 621fc8d377...

Current branch diff-target is up to date.
Changes applied before test
commit 1683a12360653b9c58203f5eb9862ab7bfc7b54e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 15:06:00 2020 +0200

    Relax checks on journal writes regarding origin-visit*

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/343/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/343/console

ardumont edited the summary of this revision. (Show Details)Jun 24 2020, 4:41 PM
douardda accepted this revision.Jun 24 2020, 5:07 PM

It's a bit sad we cannot generically use sets for this (because some model objects are still unhashable), so fine for me.

This revision is now accepted and ready to land.Jun 24 2020, 5:07 PM

It's a bit sad we cannot generically use sets for this (because some model objects are still unhashable), so fine for me.

yeah, we'll eventually be able to rework this "soon" (hopefully).

ardumont updated this revision to Diff 11885.Jun 25 2020, 11:33 AM

Rebase on D3342 (so builds fine)

Depends on D3342

Build is green

Patch application report for D3346 (id=11885)

Could not rebase; Attempt merge onto 621fc8d377...

Updating 621fc8d3..fd240af6
Fast-forward
 requirements-swh.txt               |   2 +-
 sql/upgrades/155.sql               |  34 +++
 swh/storage/backfill.py            |  10 +-
 swh/storage/cassandra/cql.py       |   3 -
 swh/storage/cassandra/schema.py    |   3 -
 swh/storage/cassandra/storage.py   |  15 +-
 swh/storage/db.py                  |  27 ++-
 swh/storage/in_memory.py           |  18 +-
 swh/storage/sql/30-swh-schema.sql  |  11 +-
 swh/storage/sql/40-swh-func.sql    |   4 +-
 swh/storage/sql/60-swh-indexes.sql |   2 +-
 swh/storage/storage.py             |  10 +-
 swh/storage/tests/test_backfill.py |   5 +-
 swh/storage/tests/test_storage.py  | 432 ++++++++++++++++---------------------
 14 files changed, 267 insertions(+), 309 deletions(-)
 create mode 100644 sql/upgrades/155.sql
Changes applied before test
commit fd240af6234e205b4593d4a3691fd9c01cd71ac9
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 15:06:00 2020 +0200

    Relax checks on journal writes regarding origin-visit*

commit e5e80efeedecd90794e18abbf6815edb6b991a3e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 10:57:52 2020 +0200

    storage*: Drop obsolete fields from origin_visit
    
    Related to T2310

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/351/ for more details.

ardumont updated this revision to Diff 11894.Jun 25 2020, 2:35 PM

Rebase on latest master

Build is green

Patch application report for D3346 (id=11894)

Rebasing onto c6e6f33916...

Current branch diff-target is up to date.
Changes applied before test
commit 12d729b9cf3aaeb7765ca513282469f762faa243
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 24 15:06:00 2020 +0200

    Relax checks on journal writes regarding origin-visit*

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/353/ for more details.