Page MenuHomeSoftware Heritage
Differential D2791

swh-scanner: integration tests
ClosedPublic
Actions

Authored by DanSeraf on Mar 6 2020, 4:40 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDTSCNa5555dd12464: basic test for the scanner
Summary

Basic tests for the scanner tool (T2299)

The test_scanner_result is using the flask live_server fixture as a backend, it simulates the response of the Web API endpoint /known/. The flask /known/ endpoint uses a predefined list of persistent identifiers available in swh.scanner.tests.data.

Diff Detail

Repository
rDTSCN Code scanner
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

DanSeraf created this revision.Mar 6 2020, 4:40 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptMar 6 2020, 4:40 PM
swh-public-ci added a comment.Mar 6 2020, 4:40 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/1/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/1/console

Harbormaster failed remote builds in B10987: Diff 9917!Mar 6 2020, 4:40 PM
DanSeraf updated this revision to Diff 9918.Mar 6 2020, 4:44 PM

make flake8 pass

swh-public-ci added a comment.Mar 6 2020, 4:44 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/2/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/2/console

Harbormaster failed remote builds in B10988: Diff 9918!Mar 6 2020, 4:44 PM
vlorentz added a subscriber: vlorentz.Mar 6 2020, 4:46 PM

I'm restarting the tests

DanSeraf updated this revision to Diff 9919.Mar 6 2020, 4:47 PM

clean flask_api.py

swh-public-ci added a comment.Mar 6 2020, 4:47 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/3/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/3/console

DanSeraf added a comment.Mar 6 2020, 4:48 PM
In D2791#66642, @vlorentz wrote:

I'm restarting the tests

The test will fail because of D2785

swh-public-ci added a comment.Mar 6 2020, 4:48 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/4/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/4/console

Harbormaster failed remote builds in B10989: Diff 9919!Mar 6 2020, 4:48 PM
vlorentz added a comment.Mar 6 2020, 4:49 PM

You can tell Arcanist to apply both diffs before running the tests. On this page, click "Edit related revisions" > "Edit parent revisions" > add D2785

DanSeraf added a parent revision: D2785: scanner: check that pid exists before yield the sub path.Mar 6 2020, 4:51 PM
vlorentz requested changes to this revision.Mar 6 2020, 4:59 PM

Looks good! Just a couple of nitpicks:

swh/scanner/tests/test_scanner.py
16

Use "example.org" or one of its subdomains as domain, it avoids using a domain that belongs to someone.

46
def test_app(app):
    """Ensures the app not in debug mode"""
    assert not app.debug

The second argument of assert is a message to be displayed on error, not a comment.

This revision now requires changes to proceed.Mar 6 2020, 4:59 PM
DanSeraf updated this revision to Diff 9921.Mar 6 2020, 5:08 PM

rebase

DanSeraf marked 2 inline comments as done.Mar 6 2020, 5:08 PM
swh-public-ci added a comment.Mar 6 2020, 5:09 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/6/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/6/console

Harbormaster failed remote builds in B10991: Diff 9921!Mar 6 2020, 5:09 PM
DanSeraf updated this revision to Diff 9928.Mar 9 2020, 3:19 PM

rebase

swh-public-ci added a comment.Mar 9 2020, 3:20 PM

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/8/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tox/8/console

Harbormaster failed remote builds in B10999: Diff 9928!Mar 9 2020, 3:20 PM
DanSeraf updated this revision to Diff 9936.Mar 9 2020, 5:50 PM

make tox pass

swh-public-ci added a comment.Mar 9 2020, 5:51 PM

Build is green
See https://jenkins.softwareheritage.org/job/DTSCN/job/tox/9/ for more details.

Harbormaster completed remote builds in B11007: Diff 9936.Mar 9 2020, 5:51 PM
zack added a subscriber: zack.Mar 9 2020, 6:20 PM
In D2791#66794, @swh-public-ci wrote:

Build is green
See https://jenkins.softwareheritage.org/job/DTSCN/job/tox/9/ for more details.

\o/ congrats @DanSeraf !

vlorentz accepted this revision.Mar 10 2020, 10:39 AM
This revision is now accepted and ready to land.Mar 10 2020, 10:39 AM
Closed by commit rDTSCNa5555dd12464: basic test for the scanner (authored by DanSeraf). · Explain WhyMar 10 2020, 10:46 AM
This revision was automatically updated to reflect the committed changes.
DanSeraf added a commit: rDTSCNa5555dd12464: basic test for the scanner.

Revision Contents
Changeset List

PathSize
1 line
1 line
4 lines
swh/
scanner/
tests/
68 lines
19 lines
data/
1 line
sample-folder/
bar/
barfoo/
2 lines
barfoo2/
1 line
foo/
1 line
1 line
1 line
1 line
1 line
1 line
24 lines
4 lines
65 lines

Diff 9939

View Options

MANIFEST.in

Loading...
View Options

pytest.ini

Loading...
View Options

requirements-test.txt

Loading...
View Options

swh/scanner/tests/conftest.py

Loading...
View Options

swh/scanner/tests/data.py

Loading...
View Options

swh/scanner/tests/data/sample-folder-result.json

Loading...
View Options

swh/scanner/tests/data/sample-folder/bar/barfoo/another-quote.org

Loading...
View Options

swh/scanner/tests/data/sample-folder/bar/barfoo2/some-file.txt

Loading...
View Options

swh/scanner/tests/data/sample-folder/foo/barfoo

Loading...
View Options

swh/scanner/tests/data/sample-folder/foo/quotes.md

Loading...
View Options

swh/scanner/tests/data/sample-folder/foo/rel-link-to-barfoo

Loading...
View Options

swh/scanner/tests/data/sample-folder/link-to-another-quote

Loading...
View Options

swh/scanner/tests/data/sample-folder/link-to-foo

Loading...
View Options

swh/scanner/tests/data/sample-folder/some-binary

Loading...
View Options

swh/scanner/tests/flask_api.py

Loading...
View Options

swh/scanner/tests/test_model.py

Loading...
View Options

swh/scanner/tests/test_scanner.py

Loading...
About · Developer information · Legal