Page MenuHomeSoftware Heritage

Add new `swh save submit-request` cli to batch save code now requests
ClosedPublic

Authored by ardumont on Apr 20 2021, 10:58 AM.

Details

Summary

Related to T3278

Test Plan

rescheduled the pending running tasks described in the task

details: T3278#63784

tox

Diff Detail

Repository
rDWCLI Web client
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 20898
Build 32429: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 32428: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D5559 (id=19856)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 938455d31b9e30ef6d1d8fffb4b4f1af02228fa4
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/40/ for more details.

ardumont edited the test plan for this revision. (Show Details)

This would indeed be useful to have the endpoint wrapped by swh-web-client.

It would be nice to also have the GET request wrapped for the save code now endpoint.

In that case, info about previously submitted requests for a given origin will be returned.

swh/web/client/cli.py
129

maybe call the command simply save ?

141

command could be named submit-request instead.

swh web save submit-request

After discussing with anlambert, a more appropriate use could be:

 tail -10 /tmp/reschedule-save-code-now3.csv | swh web save submit-request | jq .                                                              WARNING:swh.core.cli:Could not load subcommand scanner: swh-scanner
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair-telemetry)
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair-parts)
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair-casing)
INFO:root:Submitted origin (git, https://github.com/makers-for-life/makair)
INFO:root:Submitted origin (git, https://github.com/prestodb/presto-admin)
INFO:root:Submitted origin (git, https://github.com/hidekatsu-izuno/wmf2svg)
INFO:root:Submitted origin (git, https://gitlab.inria.fr/pepper/riot-demo-apps)
INFO:root:Submitted origin (git, https://github.com/betagouv/ecolab-calculateurco2inverse)
INFO:root:Submitted origin (git, https://forgemia.inra.fr/agriterix/agriterix)
INFO:root:Submitted origin (git, https://dev.ds-servers.com/sip/engine)
[
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair-telemetry",
    "save_request_date": "2021-04-20T10:47:04.528459+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair-parts",
    "save_request_date": "2021-04-20T10:47:04.801410+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair-casing",
    "save_request_date": "2021-04-20T10:47:05.057137+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/makers-for-life/makair",
    "save_request_date": "2021-04-20T10:47:05.302209+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/prestodb/presto-admin",
    "save_request_date": "2021-04-20T10:47:05.558541+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/hidekatsu-izuno/wmf2svg",
    "save_request_date": "2021-04-20T10:47:06.004204+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://gitlab.inria.fr/pepper/riot-demo-apps",
    "save_request_date": "2021-04-20T10:47:06.293276+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://github.com/betagouv/ecolab-calculateurco2inverse",
    "save_request_date": "2021-04-20T10:47:06.562878+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://forgemia.inra.fr/agriterix/agriterix",
    "save_request_date": "2021-04-20T10:47:06.779423+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  },
  {
    "visit_type": "git",
    "origin_url": "https://dev.ds-servers.com/sip/engine",
    "save_request_date": "2021-04-20T10:47:07.042652+00:00",
    "save_request_status": "accepted",
    "save_task_status": "not yet scheduled",
    "visit_date": null
  }
]
swh/web/client/cli.py
129

sure

Adapt according to review and discussion

Build is green

Patch application report for D5559 (id=19866)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit b0f0863a3867a755066cf4fa0b265d93df32b68f
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/41/ for more details.

swh/web/client/cli.py
166

give some feedback to the user to show submissions are happening...

swh/web/client/cli.py
132

Save code now Subcommand to ...

  • Adjust docstring beginning
  • Start adding test on new client endpoint

Build is green

Patch application report for D5559 (id=19875)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 113ddff4e4816df09fa29d262d53119b8620b1f7
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/42/ for more details.

Actually update with the test this time...

Build is green

Patch application report for D5559 (id=19876)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 2ed4694adc0838ee90d5bb1325a108bb85897bee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Allow to batch save code now request through cli
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/43/ for more details.

ardumont retitled this revision from Allow to batch save code now request through cli to Add new `swh save submit-request` cli to batch save code now requests.Apr 20 2021, 6:32 PM

Add test on the new cli part

Build is green

Patch application report for D5559 (id=19882)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit eee49e4baabb322fb8568483b7266a850b325648
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Add new `swh save submit-request` cli to batch save code now requests
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/44/ for more details.

Looks good to me.

swh/web/client/client.py
627–628

return r.json()

This revision is now accepted and ready to land.Apr 21 2021, 11:15 AM

Build is green

Patch application report for D5559 (id=19890)

Rebasing onto de2c707eb8...

Current branch diff-target is up to date.
Changes applied before test
commit 7c8433cb92dfb8b998e0ccce415290edb702a337
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Apr 20 10:44:57 2021 +0200

    Add new `swh save submit-request` cli to batch save code now requests
    
    Related to T3278

See https://jenkins.softwareheritage.org/job/DWCLI/job/tests-on-diff/45/ for more details.