Page MenuHomeSoftware Heritage

save code now: Simplify the request status update computation
ClosedPublic

Authored by ardumont on Jun 18 2021, 2:58 PM.

Details

Summary

This tries to maximize the use of the visit status to determine the save code now
request final status (succeeded or failed). When such data is not yet available, the
task is probably not yet scheduled, scheduled or currently running. So, it fallbacks to
use the scheduler task/task_run to deduce such non-final status (e.g
scheduled/running/not yet scheduled/...).

Computing the save code now status that way will decrease the potential discrepancy [1]
between the status displayed and the origin not being browsable yet.

[1] Most likely due to some replication lag

Related to T3378

Diff Detail

Repository
rDWAPPS Web applications
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 22170
Build 34504: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 34503: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D5896 (id=21136)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit d61b8a0324b30be3852ac30f68785e1882419bee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    save code now: Simplify the request status update computation
    
    This tries to maximize the use of the visit status to determine the save code now
    request's final status (succeeded or failed). When such data is not available, it's most
    likely the task is not yet scheduled, scheduled or currently running. So, it then
    fallbacks to try and use the scheduler task/task_run to deduce such non-final
    status (e.g scheduled/running/not yet scheduled/...).
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/884/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/884/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 18 2021, 3:09 PM
Harbormaster failed remote builds in B22123: Diff 21136!
  • Adapt tests a bit (not exhaustive yet)

Build has FAILED

Patch application report for D5896 (id=21183)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 9d6144aaacc55a65c870f11b18a75daa06d77fce
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 21 18:13:15 2021 +0200

    Adapt tests a bit

commit d61b8a0324b30be3852ac30f68785e1882419bee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    save code now: Simplify the request status update computation
    
    This tries to maximize the use of the visit status to determine the save code now
    request's final status (succeeded or failed). When such data is not available, it's most
    likely the task is not yet scheduled, scheduled or currently running. So, it then
    fallbacks to try and use the scheduler task/task_run to deduce such non-final
    status (e.g scheduled/running/not yet scheduled/...).
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/886/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/886/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 21 2021, 6:26 PM
Harbormaster failed remote builds in B22170: Diff 21183!
ardumont retitled this revision from wip: save code now: Simplify the request status update computation to save code now: Simplify the request status update computation.Jun 24 2021, 3:25 PM
ardumont edited the summary of this revision. (Show Details)
ardumont edited the test plan for this revision. (Show Details)
ardumont added a project: Save Code Now.
ardumont edited the summary of this revision. (Show Details)
  • Rebase
  • amend commit message
  • update tests

Build is green

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/890/ for more details.

Build was aborted

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/891/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/891/console

Build has FAILED

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/893/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/893/console

Looks good to me ! Thanks, the save code now handling code definitely needed some cleanup after all those years.

swh/web/common/origin_save.py
344

s/must be in a final status/must be in a final one/ to avoid repetition

swh/web/tests/common/test_origin_save.py
597–602

To remove, no ?

661

s/And/An/

This revision is now accepted and ready to land.Jun 25 2021, 11:28 AM

Build is green

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/895/ for more details.

ardumont marked 2 inline comments as done.

Adapt according to review

swh/web/tests/common/test_origin_save.py
597–602

quite ;)

Build is green

Patch application report for D5896 (id=21273)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit f9f1e969e9d7497904f045479ee5e2a26748ff1d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 25 14:08:18 2021 +0200

    test_origin_save: Call variable status instead of hard-coding values

commit 86a9acb57c0abc569d8e3991deda3935c12560ca
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/904/ for more details.