Page MenuHomeSoftware Heritage
Differential D5559

Add new `swh save submit-request` cli to batch save code now requests
ClosedPublic
Actions

Authored by ardumont on Apr 20 2021, 10:58 AM.

Details

Reviewers
anlambert
Group Reviewers
Reviewers
Maniphest Tasks
T3278: Check older pending save code now requests apparently stuck and reschedule those
Commits
rDWCLI7c8433cb92df: Add new `swh save submit-request` cli to batch save code now requests
Summary

Related to T3278

Test Plan

rescheduled the pending running tasks described in the task

details: T3278#63784

tox

Diff Detail

Repository
rDWCLI Web client
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 20865
Build 32378: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 32377: arc lint + arc unit

Event Timeline

ardumont created this revision.Apr 20 2021, 10:58 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptApr 20 2021, 10:58 AM
ardumont mentioned this in T3278: Check older pending save code now requests apparently stuck and reschedule those.Apr 20 2021, 10:58 AM
swh-public-ci added a comment.Apr 20 2021, 10:59 AM

Build is green

Patch application report for D5559 (id=19856)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 938455d31b9e30ef6d1d8fffb4b4f1af02228fa4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/40/ for more details.

Harbormaster completed remote builds in B20865: Diff 19856.Apr 20 2021, 10:59 AM
ardumont requested review of this revision.Apr 20 2021, 10:59 AM
ardumont edited the summary of this revision. (Show Details)Apr 20 2021, 11:00 AM
ardumont edited the test plan for this revision. (Show Details)
ardumont edited the test plan for this revision. (Show Details)Apr 20 2021, 11:05 AM
ardumont edited the summary of this revision. (Show Details)Apr 20 2021, 11:17 AM
anlambert added a subscriber: anlambert.Apr 20 2021, 11:47 AM

This would indeed be useful to have the endpoint wrapped by swh-web-client.

It would be nice to also have the GET request wrapped for the save code now endpoint.

In that case, info about previously submitted requests for a given origin will be returned.

swh/web/client/cli.py
129

maybe call the command simply save ?

141

command could be named submit-request instead.

swh web save submit-request

ardumont added a comment.Apr 20 2021, 12:48 PM

After discussing with anlambert, a more appropriate use could be:

 tail -10 /tmp/reschedule-save-code-now3.csv | swh web save submit-request | jq .                                                              WARNING:swh.core.cli:Could not load subcommand scanner: swh-scanner
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair-telemetry)
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair-parts)
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair-casing)
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair)
INFO:root:Submitted origin (git, https://github.com/prestodb/presto-admin)
INFO:root:Submitted origin (git, https://github.com/hidekatsu-izuno/wmf2svg)
INFO:root:Submitted origin (git, https://gitlab.inria.fr/pepper/riot-demo-apps)
INFO:root:Submitted origin (git, https://github.com/betagouv/ecolab-calculateurco2inverse)
INFO:root:Submitted origin (git, https://forgemia.inra.fr/agriterix/agriterix)
INFO:root:Submitted origin (git, https://dev.ds-servers.com/sip/engine)
[
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair-telemetry",
    "save_request_date": "2021-04-20T10:47:04.528459+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair-parts",
    "save_request_date": "2021-04-20T10:47:04.801410+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair-casing",
    "save_request_date": "2021-04-20T10:47:05.057137+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair",
    "save_request_date": "2021-04-20T10:47:05.302209+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/prestodb/presto-admin",
    "save_request_date": "2021-04-20T10:47:05.558541+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/hidekatsu-izuno/wmf2svg",
    "save_request_date": "2021-04-20T10:47:06.004204+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://gitlab.inria.fr/pepper/riot-demo-apps",
    "save_request_date": "2021-04-20T10:47:06.293276+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/betagouv/ecolab-calculateurco2inverse",
    "save_request_date": "2021-04-20T10:47:06.562878+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://forgemia.inra.fr/agriterix/agriterix",
    "save_request_date": "2021-04-20T10:47:06.779423+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://dev.ds-servers.com/sip/engine",
    "save_request_date": "2021-04-20T10:47:07.042652+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  }
]
swh/web/client/cli.py
129

sure

ardumont updated this revision to Diff 19866.Apr 20 2021, 12:49 PM

Adapt according to review and discussion

swh-public-ci added a comment.Apr 20 2021, 12:49 PM

Build is green

Patch application report for D5559 (id=19866)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit b0f0863a3867a755066cf4fa0b265d93df32b68f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/41/ for more details.

Harbormaster completed remote builds in B20876: Diff 19866.Apr 20 2021, 12:49 PM
ardumont added inline comments.Apr 20 2021, 2:32 PM
swh/web/client/cli.py
165

give some feedback to the user to show submissions are happening...

anlambert added inline comments.Apr 20 2021, 2:56 PM
swh/web/client/cli.py
131

Save code now Subcommand to ...

ardumont updated this revision to Diff 19875.Apr 20 2021, 5:14 PM
  • Adjust docstring beginning
  • Start adding test on new client endpoint
ardumont edited the summary of this revision. (Show Details)Apr 20 2021, 5:14 PM
swh-public-ci added a comment.Apr 20 2021, 5:14 PM

Build is green

Patch application report for D5559 (id=19875)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 113ddff4e4816df09fa29d262d53119b8620b1f7
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/42/ for more details.

Harbormaster completed remote builds in B20886: Diff 19875.Apr 20 2021, 5:14 PM
ardumont updated this revision to Diff 19876.Apr 20 2021, 5:23 PM

Actually update with the test this time...

swh-public-ci added a comment.Apr 20 2021, 5:24 PM

Build is green

Patch application report for D5559 (id=19876)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 2ed4694adc0838ee90d5bb1325a108bb85897bee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/43/ for more details.

Harbormaster completed remote builds in B20887: Diff 19876.Apr 20 2021, 5:24 PM
ardumont retitled this revision from Allow to batch save code now request through cli to Add new `swh save submit-request` cli to batch save code now requests.Apr 20 2021, 6:32 PM
ardumont updated this revision to Diff 19882.Apr 20 2021, 6:32 PM

Add test on the new cli part

ardumont edited the test plan for this revision. (Show Details)Apr 20 2021, 6:33 PM
swh-public-ci added a comment.Apr 20 2021, 6:33 PM

Build is green

Patch application report for D5559 (id=19882)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit eee49e4baabb322fb8568483b7266a850b325648
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Add new `swh save submit-request` cli to batch save code now requests
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/44/ for more details.

Harbormaster completed remote builds in B20892: Diff 19882.Apr 20 2021, 6:33 PM
anlambert accepted this revision.Apr 21 2021, 11:15 AM

Looks good to me.

swh/web/client/client.py
627–628

return r.json()

This revision is now accepted and ready to land.Apr 21 2021, 11:15 AM
ardumont updated this revision to Diff 19890.Apr 21 2021, 11:27 AM

Adapt last comment ;)

swh-public-ci added a comment.Apr 21 2021, 11:29 AM

Build is green

Patch application report for D5559 (id=19890)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 7c8433cb92dfb8b998e0ccce415290edb702a337
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Add new `swh save submit-request` cli to batch save code now requests
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/45/ for more details.

Harbormaster completed remote builds in B20898: Diff 19890.Apr 21 2021, 11:29 AM
Closed by commit rDWCLI7c8433cb92df: Add new `swh save submit-request` cli to batch save code now requests (authored by ardumont). · Explain WhyApr 21 2021, 11:29 AM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDWCLI7c8433cb92df: Add new `swh save submit-request` cli to batch save code now requests.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
web/
client/
49 lines53%
Loading...
19 lines82%
Loading...

Diff 19856

View Options

swh/web/client/cli.py

Loading...
View Options

swh/web/client/client.py

Loading...
About · Developer information · Legal