Page MenuHomeSoftware Heritage
Differential D5896

save code now: Simplify the request status update computation
ClosedPublic
Actions

Authored by ardumont on Jun 18 2021, 2:58 PM.

Details

Reviewers
anlambert
Group Reviewers
Reviewers
Maniphest Tasks
T3378: Improve save code now status report so the status and browse experience stay consistent
Commits
rDWAPPSf9f1e969e9d7: test_origin_save: Call variable status instead of hard-coding values
rDWAPPS86a9acb57c0a: Simplify save code now request status updates using visit statuses
Summary

This tries to maximize the use of the visit status to determine the save code now
request final status (succeeded or failed). When such data is not yet available, the
task is probably not yet scheduled, scheduled or currently running. So, it fallbacks to
use the scheduler task/task_run to deduce such non-final status (e.g
scheduled/running/not yet scheduled/...).

Computing the save code now status that way will decrease the potential discrepancy [1]
between the status displayed and the origin not being browsable yet.

[1] Most likely due to some replication lag

Related to T3378

Diff Detail

Repository
rDWAPPS Web applications
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 22170
Build 34504: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 34503: arc lint + arc unit

Unit TestsFailed
View All

TimeTest
14 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.common.test_origin_save::Tests / Python tests / test_refresh_in_progress_save_request_statuses
mocker = <pytest_mock.plugin.MockerFixture object at 0x7f105c42e828> api_client = <rest_framework.test.APIClient object at 0x7f105c42e860> archive_data = <swh.web.tests.conftest._ArchiveData object at 0x7f105c022128>
25,216 msJenkins > Should display properly entries::Tests / Cypress tests / Run cypress tests / Test admin deposit page Should display properly entries
AssertionError: Timed out retrying after 20000ms: Expected not to find content: '614' but continuously found it. at Context.eval (http://localhost:5004/__cypress/tests?p=cypress/integration/deposit-admin.spec.js:206:35)
224 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::Tests / Python tests / test_accept_pending_save_request
256 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::Tests / Python tests / test_add_authorized_origin_url
122 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::Tests / Python tests / test_add_unauthorized_origin_url
View Full Test Results (2 Failed · 690 Passed · 5 Skipped)

Event Timeline

ardumont created this revision.Jun 18 2021, 2:58 PM
swh-public-ci added a comment.Jun 18 2021, 3:09 PM

Build has FAILED

Patch application report for D5896 (id=21136)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit d61b8a0324b30be3852ac30f68785e1882419bee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    save code now: Simplify the request status update computation
    
    This tries to maximize the use of the visit status to determine the save code now
    request's final status (succeeded or failed). When such data is not available, it's most
    likely the task is not yet scheduled, scheduled or currently running. So, it then
    fallbacks to try and use the scheduler task/task_run to deduce such non-final
    status (e.g scheduled/running/not yet scheduled/...).
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/884/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/884/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 18 2021, 3:09 PM
Harbormaster failed remote builds in B22123: Diff 21136!
ardumont updated this revision to Diff 21183.Jun 21 2021, 6:15 PM
  • Adapt tests a bit (not exhaustive yet)
swh-public-ci added a comment.Jun 21 2021, 6:26 PM

Build has FAILED

Patch application report for D5896 (id=21183)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 9d6144aaacc55a65c870f11b18a75daa06d77fce
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Jun 21 18:13:15 2021 +0200

    Adapt tests a bit

commit d61b8a0324b30be3852ac30f68785e1882419bee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    save code now: Simplify the request status update computation
    
    This tries to maximize the use of the visit status to determine the save code now
    request's final status (succeeded or failed). When such data is not available, it's most
    likely the task is not yet scheduled, scheduled or currently running. So, it then
    fallbacks to try and use the scheduler task/task_run to deduce such non-final
    status (e.g scheduled/running/not yet scheduled/...).
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/886/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/886/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jun 21 2021, 6:26 PM
Harbormaster failed remote builds in B22170: Diff 21183!
ardumont retitled this revision from wip: save code now: Simplify the request status update computation to save code now: Simplify the request status update computation.Jun 24 2021, 3:25 PM
ardumont edited the summary of this revision. (Show Details)
ardumont edited the test plan for this revision. (Show Details)
ardumont added a project: Save Code Now.
ardumont updated this revision to Diff 21251.Jun 24 2021, 3:25 PM
ardumont edited the summary of this revision. (Show Details)
  • Rebase
  • amend commit message
  • update tests
swh-public-ci added a comment.Jun 24 2021, 3:35 PM

Build is green

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/890/ for more details.

Harbormaster completed remote builds in B22233: Diff 21251.Jun 24 2021, 3:35 PM
ardumont requested review of this revision.Jun 24 2021, 3:35 PM
swh-public-ci added a comment.Jun 25 2021, 11:05 AM

Build was aborted

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/891/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/891/console

swh-public-ci added a comment.Jun 25 2021, 11:16 AM

Build has FAILED

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/893/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/893/console

anlambert accepted this revision.Jun 25 2021, 11:28 AM

Looks good to me ! Thanks, the save code now handling code definitely needed some cleanup after all those years.

swh/web/common/origin_save.py
344

s/must be in a final status/must be in a final one/ to avoid repetition

swh/web/tests/common/test_origin_save.py
597–602

To remove, no ?

661

s/And/An/

This revision is now accepted and ready to land.Jun 25 2021, 11:28 AM
swh-public-ci added a comment.Jun 25 2021, 11:30 AM

Build is green

Patch application report for D5896 (id=21251)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit 22a7f02782ffe54eb1a7026a0c6c2c1394837ab3
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/895/ for more details.

ardumont updated this revision to Diff 21273.Jun 25 2021, 2:15 PM
ardumont marked 2 inline comments as done.

Adapt according to review

swh/web/tests/common/test_origin_save.py
597–602

quite ;)

swh-public-ci added a comment.Jun 25 2021, 2:29 PM

Build is green

Patch application report for D5896 (id=21273)

Rebasing onto 1bd55d031e...

Current branch diff-target is up to date.
Changes applied before test
commit f9f1e969e9d7497904f045479ee5e2a26748ff1d
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 25 14:08:18 2021 +0200

    test_origin_save: Call variable status instead of hard-coding values

commit 86a9acb57c0abc569d8e3991deda3935c12560ca
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Jun 18 14:52:33 2021 +0200

    Simplify save code now request status updates using visit statuses
    
    This maximizes the use of the origin visit status to determine the save code now request
    final status (succeeded or failed). When such data is not yet available, the task is
    probably not yet scheduled, scheduled or currently running. So, it fallbacks to use the
    scheduler task/task_run to deduce such non-final statuses (e.g scheduled/running/not yet
    scheduled/...).
    
    Computing the save code now status that way will decrease the potential discrepancy [1]
    between the status displayed and the origin not being browsable yet.
    
    [1] Most likely due to some replication lag
    
    Related to T3378

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/904/ for more details.

Harbormaster completed remote builds in B22252: Diff 21273.Jun 25 2021, 2:29 PM
Closed by commit rDWAPPS86a9acb57c0a: Simplify save code now request status updates using visit statuses (authored by ardumont). · Explain WhyJun 25 2021, 2:31 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDWAPPS86a9acb57c0a: Simplify save code now request status updates using visit statuses.
ardumont added a commit: rDWAPPSf9f1e969e9d7: test_origin_save: Call variable status instead of hard-coding values.

Revision Contents
Changeset List

PathSize
swh/
web/
common/
109 lines
tests/
common/
19 lines

Diff 21183

View Options

swh/web/common/origin_save.py

Loading...
View Options

swh/web/tests/common/test_origin_save.py

Loading...
About · Developer information · Legal