scanner-benchmark: algo_min fixed, retry mechanism on request error
ClosedPublic
Actions

Authored by DanSeraf on Jan 26 2021, 12:26 PM.

Details

Reviewers

Group Reviewers

Commits

rDTSCN5e01c09af4c6: retry request in case of backend failure
rDTSCNebad16c02da6: algo_min: remove the current node as well

Diff Detail

Repository

rDTSCN Code scanner

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

DanSeraf created this revision.Jan 26 2021, 12:26 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJan 26 2021, 12:26 PM

Build has FAILED

Patch application report for D4947 (id=17605)

Could not rebase; Attempt merge onto 33a9cd4eb9...

Auto-merging swh/scanner/cli.py
Merge made by the 'recursive' strategy.
 benchmark.py                   | 136 ++++++++++++++
 run_backend.sh                 |  15 ++
 run_benchmark.sh               |  37 ++++
 swh/scanner/backend.py         |  16 +-
 swh/scanner/benchmark_algos.py | 392 +++++++++++++++++++++++++++++++++++++++++
 swh/scanner/cli.py             |  73 ++++++++
 swh/scanner/model.py           |  54 +++++-
 7 files changed, 712 insertions(+), 11 deletions(-)
 create mode 100755 benchmark.py
 create mode 100755 run_backend.sh
 create mode 100755 run_benchmark.sh
 create mode 100644 swh/scanner/benchmark_algos.py

Changes applied before test

commit 99acd6060774103e9487b6d546d10b04deed4b9f
Merge: 33a9cd4 dbd136e
Author: Jenkins user <jenkins@localhost>
Date:   Tue Jan 26 11:26:28 2021 +0000

    Merge branch 'diff-target' into HEAD

commit dbd136e4dfbf92c69f83502d757f9cde9ecd1aba
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 26 10:10:00 2021 +0100

    retry mechanism in case of backend failure

commit a0b83c48f3f1714b1b4605657ec63d994a6ec0e5
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Mon Jan 25 13:48:14 2021 +0100

    the current node in algo_min must be removed

commit 5cd9f762467ece41d7d8e1ae1841e1d24aad45e4
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Mon Jan 18 10:26:06 2021 +0100

    fix: the temporary directory is removed by tempfile

commit 7a289332f73025f94f7f85ab5bd6755b876ebe68
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 12 23:12:18 2021 +0100

    print results as a csv

commit 9e4df16d9486a891498124dd4cfb7558c57dfa0c
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 12 23:10:39 2021 +0100

    extract repositories in temporary directories

commit 7bd1939949dcbcf0c52b8647f2b1750f2c9d2300
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Thu Dec 10 23:59:31 2020 +0100

    scanner experiments

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/91/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/91/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 26 2021, 12:28 PM

Harbormaster failed remote builds in B18744: Diff 17605!

DanSeraf requested review of this revision.Jan 26 2021, 12:42 PM

Nitpick: could you reword the commit to follow this guideline?

It would be nice to have tests for this file, but I guess that's out of scope for this diff.

Do you know why unrelated tests are failing?

for context: this change is only going to be committed in a separate branch that we're using for experiments with swh-scanner, hence the testing requirements are different/lower than for master

DanSeraf retitled this revision from scanner-benchmark: algo_mix fixed, retry mechanism on request error to scanner-benchmark: algo_miv fixed, retry mechanism on request error.Jan 26 2021, 3:15 PM

DanSeraf retitled this revision from scanner-benchmark: algo_miv fixed, retry mechanism on request error to scanner-benchmark: algo_min fixed, retry mechanism on request error.

oh, that explains the test failures, thanks

commit message

Build has FAILED

Patch application report for D4947 (id=17628)

Could not rebase; Attempt merge onto 33a9cd4eb9...

Auto-merging swh/scanner/cli.py
Merge made by the 'recursive' strategy.
 benchmark.py                   | 136 ++++++++++++++
 run_backend.sh                 |  15 ++
 run_benchmark.sh               |  37 ++++
 swh/scanner/backend.py         |  16 +-
 swh/scanner/benchmark_algos.py | 392 +++++++++++++++++++++++++++++++++++++++++
 swh/scanner/cli.py             |  73 ++++++++
 swh/scanner/model.py           |  54 +++++-
 7 files changed, 712 insertions(+), 11 deletions(-)
 create mode 100755 benchmark.py
 create mode 100755 run_backend.sh
 create mode 100755 run_benchmark.sh
 create mode 100644 swh/scanner/benchmark_algos.py

Changes applied before test

commit 3aef643dfeef93141977f45a62fde771adaadf4e
Merge: 33a9cd4 5e01c09
Author: Jenkins user <jenkins@localhost>
Date:   Tue Jan 26 14:46:05 2021 +0000

    Merge branch 'diff-target' into HEAD

commit 5e01c09af4c61a309d71adb0d4f61d1766b8a021
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 26 10:10:00 2021 +0100

    retry request in case of backend failure

commit ebad16c02da6bffbc96a623e082a4b5f706d7b1f
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Mon Jan 25 13:48:14 2021 +0100

    algo_min: remove the current node as well

commit 5cd9f762467ece41d7d8e1ae1841e1d24aad45e4
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Mon Jan 18 10:26:06 2021 +0100

    fix: the temporary directory is removed by tempfile

commit 7a289332f73025f94f7f85ab5bd6755b876ebe68
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 12 23:12:18 2021 +0100

    print results as a csv

commit 9e4df16d9486a891498124dd4cfb7558c57dfa0c
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Tue Jan 12 23:10:39 2021 +0100

    extract repositories in temporary directories

commit 7bd1939949dcbcf0c52b8647f2b1750f2c9d2300
Author: Daniele Serafini <me@danieleserafini.eu>
Date:   Thu Dec 10 23:59:31 2020 +0100

    scanner experiments

Link to build: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/92/
See console output for more information: https://jenkins.softwareheritage.org/job/DTSCN/job/tests-on-diff/92/console

Harbormaster failed remote builds in B18766: Diff 17628!Jan 26 2021, 3:48 PM

zack accepted this revision.Jan 28 2021, 10:58 AM

This revision is now accepted and ready to land.Jan 28 2021, 10:58 AM

This revision was landed with ongoing or failed builds.Jan 28 2021, 11:44 AM

Closed by commit rDTSCNebad16c02da6: algo_min: remove the current node as well (authored by DanSeraf). · Explain Why

This revision was automatically updated to reflect the committed changes.

DanSeraf added a commit: rDTSCNebad16c02da6: algo_min: remove the current node as well.

DanSeraf added a commit: rDTSCN5e01c09af4c6: retry request in case of backend failure.