as this happens quite often and isn't an error.
Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Commits
- rDSTO3eba73df9f38: migrate_extrinsic_metadata: Remove log output when a CRAN origin is missing
Diff Detail
Diff Detail
- Repository
- rDSTO Storage manager
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D4437 (id=15704)
Could not rebase; Attempt merge onto 84984a600c...
Updating 84984a60..c83c9020 Fast-forward requirements-swh-journal.txt | 2 +- swh/storage/backfill.py | 10 +- swh/storage/migrate_extrinsic_metadata.py | 21 +- swh/storage/retry.py | 80 +-- .../tests/migrate_extrinsic_metadata/test_pypi.py | 2 + swh/storage/tests/test_retry.py | 640 --------------------- swh/storage/writer.py | 21 +- 7 files changed, 62 insertions(+), 714 deletions(-)
Changes applied before test
commit c83c9020f0e6d6ef634972c5e85b2345e00118cc Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:57:02 2020 +0100 migrate_extrinsic_metadata: Remove log output when a CRAN origin is missing as this happens quite often and isn't an error. commit 0c8000145778d1774fede5241b0d6dbf28d9aecb Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:55:58 2020 +0100 migrate_extrinsic_metadata: add support for guessing the origin of more PyPI packages from filenames. commit fc46b7f277603d66c4ae80294cdd4e3d32125949 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:54:14 2020 +0100 migrate_extrinsic_metadata: use the retry proxy Because it makes a lot of get requests and doesn't handle failures, it crashed often. commit 5794e0403f2ad69c9ccd98733bceee908d1bf21e Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:52:50 2020 +0100 Make the retry proxy work on all functions. The metadata migration script kept crashing otherwise. commit 9d1237b4eae1d64ca784e3ebb7f0f390e551b3a6 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Thu Nov 5 15:28:49 2020 +0100 Set the value_sanitizer argument of get_journal_writer. The next version of swh-journal will remove the default value.
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1055/ for more details.
Comment Actions
Build is green
Patch application report for D4437 (id=15756)
Could not rebase; Attempt merge onto 24cdc85c15...
Updating 24cdc85c..3eba73df Fast-forward requirements-swh-journal.txt | 2 +- swh/storage/backfill.py | 10 +- swh/storage/migrate_extrinsic_metadata.py | 21 +- swh/storage/retry.py | 80 +-- .../tests/migrate_extrinsic_metadata/test_pypi.py | 2 + swh/storage/tests/test_retry.py | 640 --------------------- swh/storage/writer.py | 21 +- 7 files changed, 62 insertions(+), 714 deletions(-)
Changes applied before test
commit 3eba73df9f38beb5e35e35c5d56d32ae87372184 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:57:02 2020 +0100 migrate_extrinsic_metadata: Remove log output when a CRAN origin is missing as this happens quite often and isn't an error. commit f3652a97118c1bbfea6328ec1d9f913941b288f9 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:55:58 2020 +0100 migrate_extrinsic_metadata: add support for guessing the origin of more PyPI packages from filenames. commit c0a3d966faf45f4623c7e1bc03f5fa72024d22cf Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:54:14 2020 +0100 migrate_extrinsic_metadata: use the retry proxy Because it makes a lot of get requests and doesn't handle failures, it crashed often. commit aded45b9b27d93df8195fde5f462250e2e84798b Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 6 12:52:50 2020 +0100 Make the retry proxy work on all functions. The metadata migration script kept crashing otherwise. commit 2e7d489eb245d4c3b684360d6f70f061804f77a9 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Thu Nov 5 15:28:49 2020 +0100 Set the value_sanitizer argument of get_journal_writer. The next version of swh-journal will remove the default value.
See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1063/ for more details.