Page MenuHomeSoftware Heritage

Solve uneventful/eventful with unordered messages with snapshots
ClosedPublic

Authored by vsellier on Jan 21 2021, 7:06 PM.

Details

Summary

Fix the case:
m1: date2/snapshot1
m2: date1/snaptshot1
which results to:
last_eventful = date2
last_uneventful = date2

The upsert was always keeping the most recent date when the
eventful/uneventful dates were switched

Related to T2978

Diff Detail

Repository
rDSCH Scheduling utilities
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont added inline comments.
swh/scheduler/tests/test_scheduler.py
1000

For those wondering, that test was specifically testing the upsert part which did the max computations (and so on).

That part has been done client side for a while and now it revealed to be buggy, so we dropped it entirely.

Build is green

Patch application report for D4918 (id=17498)

Could not rebase; Attempt merge onto 03460207a1...

Auto-merging swh/scheduler/tests/test_scheduler.py
Merge made by the 'recursive' strategy.
 swh/scheduler/backend.py                   | 35 ++-----------
 swh/scheduler/journal_client.py            |  7 +++
 swh/scheduler/tests/test_journal_client.py | 81 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 60 ----------------------
 4 files changed, 93 insertions(+), 90 deletions(-)
Changes applied before test
commit 32c92fce670e83d8089344ed3023cc3842c2abca
Merge: 0346020 2550d25
Author: Jenkins user <jenkins@localhost>
Date:   Thu Jan 21 18:08:37 2021 +0000

    Merge branch 'diff-target' into HEAD

commit 2550d25568ed7538beaf077e936044d482be8d2a
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Thu Jan 21 18:49:38 2021 +0100

    Solve uneventful/eventful with unordered messages with snapshots
    
    Fix the case:
    m1: date2/snapshot1
    m2: date1/snaptshot1
    which results to:
    last_eventful = date2
    last_uneventful = date2
    
    The upsert was always keeping the most recent date when the
    eventful/uneventful dates were switched
    
    Related to T2978

commit 91cfef26c00226d34ddf09900329bb7d7c295dd3
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Thu Jan 21 16:15:26 2021 +0100

    Do not consider duplicated messages as uneventful event
    
    Avoid to copy the eventful date to the uneventful date when a
    duplicated message (same date/same snapshot) is received,
    
    related to T2978

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/241/ for more details.

This revision is now accepted and ready to land.Jan 21 2021, 7:11 PM

Build is green

Patch application report for D4918 (id=17500)

Could not rebase; Attempt merge onto 03460207a1...

Updating 0346020..82b7a8a
Fast-forward
 swh/scheduler/backend.py                   | 35 ++-----------
 swh/scheduler/journal_client.py            |  7 +++
 swh/scheduler/tests/test_journal_client.py | 81 ++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py      | 60 ----------------------
 4 files changed, 93 insertions(+), 90 deletions(-)
Changes applied before test
commit 82b7a8a42367d2286331e851462ebf5491ce0b70
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Thu Jan 21 18:49:38 2021 +0100

    Solve uneventful/eventful with unordered messages with snapshots
    
    Fix the case:
    m1: date2/snapshot1
    m2: date1/snaptshot1
    which results to:
    last_eventful = date2
    last_uneventful = date2
    
    The upsert was always keeping the most recent date when the
    eventful/uneventful dates were switched
    
    Related to T2978

commit 25d036ef45033108389d516226c2bf580714dff0
Author: Vincent SELLIER <vincent.sellier@softwareheritage.org>
Date:   Thu Jan 21 16:15:26 2021 +0100

    Do not consider duplicated messages as uneventful event
    
    Avoid to copy the eventful date to the uneventful date when a
    duplicated message (same date/same snapshot) is received,
    
    related to T2978

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/243/ for more details.