Page MenuHomeSoftware Heritage

journal_client: Only check last_* fields for some permutation tests
ClosedPublic

Authored by ardumont on Jun 23 2021, 5:04 PM.

Details

Summary

In future commits [1], we will add new fields whose values will be permutation dependent.

[1] see D5919

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D5917 (id=21240)

Could not rebase; Attempt merge onto ed818702c4...

Updating ed81870..98f99b9
Fast-forward
 swh/scheduler/backend.py                   | 13 +++++++------
 swh/scheduler/cli/task.py                  |  6 ++++--
 swh/scheduler/journal_client.py            | 21 +++++++++++----------
 swh/scheduler/tests/test_journal_client.py | 22 ++++++++++------------
 4 files changed, 32 insertions(+), 30 deletions(-)
Changes applied before test
commit 98f99b9fd457820dc2d4b5dab7e89cb8261a34a4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 16:39:40 2021 +0200

    journal_client: Only check last_* fields for some permutation tests
    
    In a future commit, we will add new fields whose values will be permutation dependent.

commit 1006f0aee494b96e333eef13331d0fad1fcc0b6e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 15:37:40 2021 +0200

    journal_client: Auto-generate the empty object from model fields
    
    This will help us when adding new fields to the table.

commit 6400cc2b95cb1afdfc54594ea2c059b69c78f6ef
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 15:28:55 2021 +0200

    backend: Auto-generate origin visit stats upsert query
    
    This will help us when adding new fields to the table.

commit 3762c34024027dd4377302736bc804ce8a6da8b0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 16:49:12 2021 +0200

    cli/task: Ensure cli output is always in the same order

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/384/ for more details.

I think I'd rather like to have an explicit list of excluded fields (when these extra fields are added). So I'd prefer see this diff be something that compares dicts (as a result of BaseObject.to_dict()), possibly filtered to exclude some fields.

This revision now requires changes to proceed.Jun 25 2021, 11:22 AM

I think I'd rather like to have an explicit list of excluded fields (when these extra fields are added). So I'd prefer see this diff be something that compares dicts (as a result of BaseObject.to_dict()), possibly filtered to exclude some fields.

Except there is no to_dict implementation in here...

[1]

        expected_visit_stats_d = {
            "url": "cavabarder",
            "visit_type": "hg",
            "last_eventful": DATE1 + 2 * ONE_DAY,
            "last_uneventful": DATE1 + 3 * ONE_DAY,
            "last_failed": None,
            "last_notfound": None,
            "last_snapshot": hash_to_bytes("aaaaaabbbeb6cf9efd5b920a8453e1e07157b6cd"),
        }

        actual_visit_stats = swh_scheduler.origin_visit_stats_get([("cavabarder", "hg")])

        assert len(actual_visit_stats) == 1
        for visit_stats in actual_visit_stats:
>           visit_stats_d = visit_stats.to_dict()
E           AttributeError: 'OriginVisitStats' object has no attribute 'to_dict'

heads up, a gazillion diffs built on this, so it will probably land like this (started
from D5919 onward). And then this will get rewritten more properly after those diffs
land (on those diffs, a new utility function appeared to actually compare the results
"appropriately" depending on the context).

olasd added a subscriber: olasd.

Ack (with the view that this will be refactored away in a few diffs' time)

This revision was not accepted when it landed; it landed in state Needs Revision.Jul 22 2021, 2:22 PM
This revision was automatically updated to reflect the committed changes.