Differential D4851

scanner benchmark: output format and repository extraction in temporary directories
ClosedPublic
Actions

Authored by DanSeraf on Jan 13 2021, 11:05 AM.

Details

Reviewers

zack

Group Reviewers

Reviewers

Commits

rDTSCN7a289332f730: print results as a csv
rDTSCN9e4df16d9486: extract repositories in temporary directories

Diff Detail

Repository

rDTSCN Code scanner

Branch

benchmark

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 18314
Build 28282: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 28281: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	156 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.scanner.tests.test_backend::test_backend_endpoint_all_present tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_backend_endpoint_all_pres0') live_server = <LiveServer listening at http://localhost:6600> test_swhids_sample = <_io.TextIOWrapper name='/tmp/pytest-of-jenkins/pytest-0/test_backend_endpoint_all_pres0/swhids_sample.txt' mode='r' encoding='UTF-8'>
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.scanner.tests.test_backend::test_backend_endpoint_one_not_present tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_backend_endpoint_one_not_0') live_server = <LiveServer listening at http://localhost:6600> test_swhids_sample = <_io.TextIOWrapper name='/tmp/pytest-of-jenkins/pytest-0/test_backend_endpoint_one_not_0/swhids_sample.txt' mode='r' encoding='UTF-8'>
	4 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.scanner.tests.test_backend::test_backend_large_payload_exc tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_backend_large_payload_exc0') live_server = <LiveServer listening at http://localhost:6600> test_swhids_sample = <_io.TextIOWrapper name='/tmp/pytest-of-jenkins/pytest-0/test_backend_large_payload_exc0/swhids_sample.txt' mode='r' encoding='UTF-8'>
	3 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.scanner.tests.test_dashboard::test_generate_table_body example_tree = <swh.scanner.model.Tree object at 0x7f3b2710e4e0> temp_folder = {'filesample': PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_generate_table_body0/subdir/filesample.txt'), 'filesamp...b9a060e54bf8d69288fbee4904', ...}, 'root': PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_generate_table_body0'), ...}
	2 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.scanner.tests.test_model::test_get_directories_info example_tree = <swh.scanner.model.Tree object at 0x7f3b27107470> temp_folder = {'filesample': PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_get_directories_info0/subdir/filesample.txt'), 'filesam...9a060e54bf8d69288fbee4904', ...}, 'root': PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_get_directories_info0'), ...}
		View Full Test Results (17 Failed · 18 Passed)

Event Timeline

DanSeraf created this revision.Jan 13 2021, 11:05 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptJan 13 2021, 11:05 AM

Build has FAILED

Patch application report for D4851 (id=17170)

Could not rebase; Attempt merge onto 33a9cd4eb9...

Auto-merging swh/scanner/cli.py
Merge made by the 'recursive' strategy.
 benchmark.py                   | 139 +++++++++++++++
 run_backend.sh                 |  15 ++
 run_benchmark.sh               |  37 ++++
 swh/scanner/backend.py         |  16 +-
 swh/scanner/benchmark_algos.py | 385 +++++++++++++++++++++++++++++++++++++++++
 swh/scanner/cli.py             |  73 ++++++++
 swh/scanner/model.py           |  54 +++++-
 7 files changed, 708 insertions(+), 11 deletions(-)
 create mode 100755 benchmark.py
 create mode 100755 run_backend.sh
 create mode 100755 run_benchmark.sh
 create mode 100644 swh/scanner/benchmark_algos.py

Changes applied before test

commit b1cc42ca424f6cd82dec69f736558afad99d0a4c
Merge: 33a9cd4 7a28933
Author: Jenkins user <jenkins@localhost>
Date:   Wed Jan 13 10:05:16 2021 +0000

    Merge branch 'diff-target' into HEAD

commit 7a289332f73025f94f7f85ab5bd6755b876ebe68
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 12 23:12:18 2021 +0100

    print results as a csv

commit 9e4df16d9486a891498124dd4cfb7558c57dfa0c
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 12 23:10:39 2021 +0100

    extract repositories in temporary directories

commit 7bd1939949dcbcf0c52b8647f2b1750f2c9d2300
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Thu Dec 10 23:59:31 2020 +0100

    scanner experiments

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/89/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/89/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 13 2021, 11:07 AM

Harbormaster failed remote builds in B18314: Diff 17170!

DanSeraf requested review of this revision.Jan 13 2021, 11:20 AM