Page MenuHomeSoftware Heritage
Differential D6114

swh-scanner: retrieve additional information about software artifacts
ClosedPublic
Actions

Authored by DanSeraf on Aug 18 2021, 7:17 PM.

Details

Reviewers
zack
Group Reviewers
Reviewers
Commits
rDTSCN979d7c803a14: retrieve additional information about software artifacts
Summary
  • additional information about software artifacts can be specified using the CLI with the -a/--add option
  • retrieve origin information about software artifacts using swh-graph
  • move backend requests to a separate submodule (client)

Diff Detail

Repository
rDTSCN Code scanner
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

DanSeraf created this revision.Aug 18 2021, 7:17 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptAug 18 2021, 7:17 PM
swh-public-ci added a comment.Aug 18 2021, 7:19 PM

Build is green

Patch application report for D6114 (id=22119)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit 995ed01d92a7758a984b6700bbdb557a9945fac7
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/150/ for more details.

Harbormaster completed remote builds in B23076: Diff 22119.Aug 18 2021, 7:19 PM
DanSeraf requested review of this revision.Aug 18 2021, 7:19 PM
vlorentz added a subscriber: vlorentz.Aug 19 2021, 1:58 PM
vlorentz added inline comments.
swh/scanner/cli.py
150

Could you describe the extra info?

DanSeraf updated this revision to Diff 22131.Aug 19 2021, 4:10 PM

extra info description

swh-public-ci added a comment.Aug 19 2021, 4:13 PM

Build is green

Patch application report for D6114 (id=22131)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit 5016132575801c709da878567b2acf5a4039f2bf
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/151/ for more details.

Harbormaster completed remote builds in B23088: Diff 22131.Aug 19 2021, 4:13 PM
DanSeraf updated this revision to Diff 22144.Aug 21 2021, 11:18 AM

missing unit test

swh-public-ci added a comment.Aug 21 2021, 11:21 AM

Build is green

Patch application report for D6114 (id=22144)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit eb30e9b7bfbfcaade02dc2cfc51e0b74ebbec9ae
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/152/ for more details.

Harbormaster completed remote builds in B23101: Diff 22144.Aug 21 2021, 11:21 AM
DanSeraf updated this revision to Diff 22145.Aug 22 2021, 6:40 PM

fix 'get_origin' response

swh-public-ci added a comment.Aug 22 2021, 6:42 PM

Build has FAILED

Patch application report for D6114 (id=22145)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit def2584d65651da90fad60a3db08d8d6bd33e442
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/153/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/153/console

Harbormaster failed remote builds in B23102: Diff 22145!Aug 22 2021, 6:42 PM
zack requested changes to this revision.Aug 23 2021, 11:18 AM
zack added a subscriber: zack.

Nice! And I also like the refactoring out of client.py.

I'm requesting mainly changes about the UX and documentation.

The CI will need to pass though :-)

swh/scanner/cli.py
145–147

I see that "-i" is already taken, hence why you haven't used it.
I think that "a" is not very mnemonic though, I propose to use "e" (for extra) info, that is: "-e", "--extra-info", "extra_info)

150

better: "add selected additional information about known software artifacts"

swh/scanner/client.py
5

please add a module docstring, starting that this is a minimal async web client for the Software Heritage Web API

also add in there also a TODO comment stating that this could be removed when T2635 is implemented

(without this info it will not be clear why this module is needed at all)

This revision now requires changes to proceed.Aug 23 2021, 11:18 AM
DanSeraf updated this revision to Diff 22146.Aug 23 2021, 1:13 PM

requested changes

swh-public-ci added a comment.Aug 23 2021, 1:16 PM

Build is green

Patch application report for D6114 (id=22146)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit 96b3467f2d330b3f010414427843dd654bd6b3cf
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/154/ for more details.

Harbormaster completed remote builds in B23103: Diff 22146.Aug 23 2021, 1:16 PM
DanSeraf updated this revision to Diff 22147.Aug 23 2021, 1:25 PM

commit message

swh-public-ci added a comment.Aug 23 2021, 6:12 PM

Build has FAILED

Patch application report for D6114 (id=22147)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit efbac760564f4ecba09cafee3ee344cd99cae496
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -e/--extra-info option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/155/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/155/console

Harbormaster failed remote builds in B23104: Diff 22147!Aug 23 2021, 6:12 PM
swh-public-ci added a comment.Aug 24 2021, 2:05 PM

Build is green

Patch application report for D6114 (id=22147)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit efbac760564f4ecba09cafee3ee344cd99cae496
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -e/--extra-info option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/156/ for more details.

Harbormaster completed remote builds in B23104: Diff 22147.Aug 24 2021, 2:05 PM
zack accepted this revision.Aug 24 2021, 5:58 PM

LGTM, thanks! But please note that remaining "-a/--add" reference in the new docstring, which should be changed to "-e/..." for consistency. Please fix that before landing this change.

swh/scanner/cli.py
177–178

this should be -e/--extra-info now

This revision is now accepted and ready to land.Aug 24 2021, 5:58 PM
DanSeraf updated this revision to Diff 22177.Aug 24 2021, 6:45 PM

fix docstring

swh-public-ci added a comment.Aug 24 2021, 6:50 PM

Build is green

Patch application report for D6114 (id=22177)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.
Changes applied before test
commit 979d7c803a1478c1e65a6cf8a827c16a746e3aa1
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -e/--extra-info option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/157/ for more details.

Harbormaster completed remote builds in B23134: Diff 22177.Aug 24 2021, 6:50 PM
Closed by commit rDTSCN979d7c803a14: retrieve additional information about software artifacts (authored by DanSeraf). · Explain WhyAug 24 2021, 6:53 PM
This revision was automatically updated to reflect the committed changes.
DanSeraf added a commit: rDTSCN979d7c803a14: retrieve additional information about software artifacts.

Revision Contents
Changeset List

PathSize
swh/
scanner/
54 lines
98 lines
45 lines
4 lines
5 lines
112 lines
32 lines
tests/
7 lines
11 lines
58 lines
29 lines
54 lines
22 lines

Diff 22178

View Options

swh/scanner/cli.py

Loading...
View Options

swh/scanner/client.py

Loading...
View Options

swh/scanner/data.py

Loading...
View Options

swh/scanner/exceptions.py

Loading...
View Options

swh/scanner/output.py

Loading...
View Options

swh/scanner/policy.py

Loading...
View Options

swh/scanner/scanner.py

Loading...
View Options

swh/scanner/tests/data.py

Loading...
View Options

swh/scanner/tests/flask_api.py

Loading...
View Options

swh/scanner/tests/test_client.py

Loading...
View Options

swh/scanner/tests/test_data.py

Loading...
View Options

swh/scanner/tests/test_policy.py

Loading...
View Options

swh/scanner/tests/test_scanner.py

Loading...
About · Developer information · Legal