Details

Reviewers

zack

Group Reviewers

Reviewers

Commits

rDTSCN979d7c803a14: retrieve additional information about software artifacts

Summary

additional information about software artifacts can be specified using the CLI with the -a/--add option
retrieve origin information about software artifacts using swh-graph
move backend requests to a separate submodule (client)

Diff Detail

Repository

rDTSCN Code scanner

Branch

provenance-info

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 23104
Build 36033: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 36032: arc lint + arc unit

Event Timeline

DanSeraf created this revision.Aug 18 2021, 7:17 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptAug 18 2021, 7:17 PM

Build is green

Patch application report for D6114 (id=22119)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit 995ed01d92a7758a984b6700bbdb557a9945fac7
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/150/ for more details.

Harbormaster completed remote builds in B23076: Diff 22119.Aug 18 2021, 7:19 PM

DanSeraf requested review of this revision.Aug 18 2021, 7:19 PM

vlorentz added a subscriber: vlorentz.Aug 19 2021, 1:58 PM

vlorentz added inline comments.

swh/scanner/cli.py
150	Could you describe the extra info?

extra info description

Build is green

Patch application report for D6114 (id=22131)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit 5016132575801c709da878567b2acf5a4039f2bf
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/151/ for more details.

Harbormaster completed remote builds in B23088: Diff 22131.Aug 19 2021, 4:13 PM

missing unit test

Build is green

Patch application report for D6114 (id=22144)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit eb30e9b7bfbfcaade02dc2cfc51e0b74ebbec9ae
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/152/ for more details.

Harbormaster completed remote builds in B23101: Diff 22144.Aug 21 2021, 11:21 AM

fix 'get_origin' response

Build has FAILED

Patch application report for D6114 (id=22145)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit def2584d65651da90fad60a3db08d8d6bd33e442
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/153/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/153/console

Harbormaster failed remote builds in B23102: Diff 22145!Aug 22 2021, 6:42 PM

Nice! And I also like the refactoring out of client.py.

I'm requesting mainly changes about the UX and documentation.

The CI will need to pass though :-)

swh/scanner/cli.py
145–147	I see that "-i" is already taken, hence why you haven't used it. I think that "a" is not very mnemonic though, I propose to use "e" (for extra) info, that is: "-e", "--extra-info", "extra_info)
150	better: "add selected additional information about known software artifacts"
swh/scanner/client.py
5	please add a module docstring, starting that this is a minimal async web client for the Software Heritage Web API also add in there also a TODO comment stating that this could be removed when T2635 is implemented (without this info it will not be clear why this module is needed at all)

This revision now requires changes to proceed.Aug 23 2021, 11:18 AM

requested changes

Build is green

Patch application report for D6114 (id=22146)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit 96b3467f2d330b3f010414427843dd654bd6b3cf
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -a/--add option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/154/ for more details.

Harbormaster completed remote builds in B23103: Diff 22146.Aug 23 2021, 1:16 PM

commit message

Build has FAILED

Patch application report for D6114 (id=22147)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit efbac760564f4ecba09cafee3ee344cd99cae496
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -e/--extra-info option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/155/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/155/console

Harbormaster failed remote builds in B23104: Diff 22147!Aug 23 2021, 6:12 PM

Build is green

Patch application report for D6114 (id=22147)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit efbac760564f4ecba09cafee3ee344cd99cae496
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -e/--extra-info option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/156/ for more details.

Harbormaster completed remote builds in B23104: Diff 22147.Aug 24 2021, 2:05 PM

LGTM, thanks! But please note that remaining "-a/--add" reference in the new docstring, which should be changed to "-e/..." for consistency. Please fix that before landing this change.

swh/scanner/cli.py
178–179	this should be -e/--extra-info now

This revision is now accepted and ready to land.Aug 24 2021, 5:58 PM

fix docstring

Build is green

Patch application report for D6114 (id=22177)

Rebasing onto a9d0b9e2af...

Current branch diff-target is up to date.

Changes applied before test

commit 979d7c803a1478c1e65a6cf8a827c16a746e3aa1
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Wed Aug 18 19:08:38 2021 +0200

    retrieve additional information about software artifacts
    
    - additional information about software artifacts can be specified
    using the CLI with the -e/--extra-info option
    - add support to retrieve origin information using swh-graph
    - move backend requests to a separate submodule (client)

See https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/157/ for more details.

Harbormaster completed remote builds in B23134: Diff 22177.Aug 24 2021, 6:50 PM

Closed by commit rDTSCN979d7c803a14: retrieve additional information about software artifacts (authored by DanSeraf). · Explain WhyAug 24 2021, 6:53 PM

This revision was automatically updated to reflect the committed changes.

DanSeraf added a commit: rDTSCN979d7c803a14: retrieve additional information about software artifacts.

swh-scanner: retrieve additional information about software artifacts
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Patch application report for D6114 (id=22119)

Changes applied before test

Patch application report for D6114 (id=22131)

Changes applied before test

Patch application report for D6114 (id=22144)

Changes applied before test

Patch application report for D6114 (id=22145)

Changes applied before test

Patch application report for D6114 (id=22146)

Changes applied before test

Patch application report for D6114 (id=22147)

Changes applied before test

Patch application report for D6114 (id=22147)

Changes applied before test

Patch application report for D6114 (id=22177)

Changes applied before test

Revision Contents
Changeset List

Diff 22147

swh/scanner/cli.py

swh/scanner/client.py

swh/scanner/data.py

swh/scanner/exceptions.py

swh/scanner/output.py

swh/scanner/policy.py

swh/scanner/scanner.py

swh/scanner/tests/data.py

swh/scanner/tests/flask_api.py

swh/scanner/tests/test_client.py

swh/scanner/tests/test_data.py

swh/scanner/tests/test_policy.py

swh/scanner/tests/test_scanner.py

swh-scanner: retrieve additional information about software artifactsClosedPublicActions

Details

Diff Detail

Event Timeline

Patch application report for D6114 (id=22119)

Changes applied before test

Patch application report for D6114 (id=22131)

Changes applied before test

Patch application report for D6114 (id=22144)

Changes applied before test

Patch application report for D6114 (id=22145)

Changes applied before test

Patch application report for D6114 (id=22146)

Changes applied before test

Patch application report for D6114 (id=22147)

Changes applied before test

Patch application report for D6114 (id=22147)

Changes applied before test

Patch application report for D6114 (id=22177)

Changes applied before test

Revision ContentsChangeset List

Diff 22147

swh/scanner/cli.py

swh/scanner/client.py

swh/scanner/data.py

swh/scanner/exceptions.py

swh/scanner/output.py

swh/scanner/policy.py

swh/scanner/scanner.py

swh/scanner/tests/data.py

swh/scanner/tests/flask_api.py

swh/scanner/tests/test_client.py

swh/scanner/tests/test_data.py

swh/scanner/tests/test_policy.py

swh/scanner/tests/test_scanner.py

swh-scanner: retrieve additional information about software artifacts
ClosedPublic
Actions

Revision Contents
Changeset List