journal_client: Only check last_* fields for some permutation tests
ClosedPublic
Actions

Authored by ardumont on Jun 23 2021, 5:04 PM.

Details

Reviewers

douardda
olasd

Group Reviewers

Reviewers

Commits

rDSCH98f99b9fd457: journal_client: Only check last_* fields for some permutation tests

Summary

In future commits [1], we will add new fields whose values will be permutation dependent.

[1] see D5919

Diff Detail

Repository

rDSCH Scheduling utilities

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Jun 23 2021, 5:04 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJun 23 2021, 5:04 PM

Build is green

Patch application report for D5917 (id=21240)

Could not rebase; Attempt merge onto ed818702c4...

Updating ed81870..98f99b9
Fast-forward
 swh/scheduler/backend.py                   | 13 +++++++------
 swh/scheduler/cli/task.py                  |  6 ++++--
 swh/scheduler/journal_client.py            | 21 +++++++++++----------
 swh/scheduler/tests/test_journal_client.py | 22 ++++++++++------------
 4 files changed, 32 insertions(+), 30 deletions(-)

Changes applied before test

commit 98f99b9fd457820dc2d4b5dab7e89cb8261a34a4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 16:39:40 2021 +0200

    journal_client: Only check last_* fields for some permutation tests
    
    In a future commit, we will add new fields whose values will be permutation dependent.

commit 1006f0aee494b96e333eef13331d0fad1fcc0b6e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 15:37:40 2021 +0200

    journal_client: Auto-generate the empty object from model fields
    
    This will help us when adding new fields to the table.

commit 6400cc2b95cb1afdfc54594ea2c059b69c78f6ef
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 15:28:55 2021 +0200

    backend: Auto-generate origin visit stats upsert query
    
    This will help us when adding new fields to the table.

commit 3762c34024027dd4377302736bc804ce8a6da8b0
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Wed Jun 23 16:49:12 2021 +0200

    cli/task: Ensure cli output is always in the same order

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/384/ for more details.

Harbormaster completed remote builds in B22224: Diff 21240.Jun 23 2021, 5:07 PM

ardumont requested review of this revision.Jun 23 2021, 5:07 PM

I think I'd rather like to have an explicit list of excluded fields (when these extra fields are added). So I'd prefer see this diff be something that compares dicts (as a result of BaseObject.to_dict()), possibly filtered to exclude some fields.

douardda requested changes to this revision.Jun 25 2021, 11:22 AM

This revision now requires changes to proceed.Jun 25 2021, 11:22 AM

I think I'd rather like to have an explicit list of excluded fields (when these extra fields are added). So I'd prefer see this diff be something that compares dicts (as a result of BaseObject.to_dict()), possibly filtered to exclude some fields.

Except there is no to_dict implementation in here...

[1]

        expected_visit_stats_d = {
            "url": "cavabarder",
            "visit_type": "hg",
            "last_eventful": DATE1 + 2 * ONE_DAY,
            "last_uneventful": DATE1 + 3 * ONE_DAY,
            "last_failed": None,
            "last_notfound": None,
            "last_snapshot": hash_to_bytes("aaaaaabbbeb6cf9efd5b920a8453e1e07157b6cd"),
        }

        actual_visit_stats = swh_scheduler.origin_visit_stats_get([("cavabarder", "hg")])

        assert len(actual_visit_stats) == 1
        for visit_stats in actual_visit_stats:
>           visit_stats_d = visit_stats.to_dict()
E           AttributeError: 'OriginVisitStats' object has no attribute 'to_dict'

ardumont edited the summary of this revision. (Show Details)Jun 25 2021, 5:38 PM

heads up, a gazillion diffs built on this, so it will probably land like this (started
from D5919 onward). And then this will get rewritten more properly after those diffs
land (on those diffs, a new utility function appeared to actually compare the results
"appropriately" depending on the context).