Because it makes a lot of get requests and doesn't handle failures,
it crashed often.
Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Commits
- rDSTOc0a3d966faf4: migrate_extrinsic_metadata: use the retry proxy
Diff Detail
Diff Detail
- Repository
- rDSTO Storage manager
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Comment Actions
Build is green
Patch application report for D4435 (id=15702)
Could not rebase; Attempt merge onto 84984a600c...
Updating 84984a60..fc46b7f2 Fast-forward requirements-swh-journal.txt | 2 +- swh/storage/backfill.py | 10 +- swh/storage/migrate_extrinsic_metadata.py | 12 +- swh/storage/retry.py | 80 +--- swh/storage/tests/test_retry.py | 640 ------------------------------ swh/storage/writer.py | 21 +- 6 files changed, 53 insertions(+), 712 deletions(-)
Changes applied before test
commit fc46b7f277603d66c4ae80294cdd4e3d32125949
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Fri Nov 6 12:54:14 2020 +0100
migrate_extrinsic_metadata: use the retry proxy
Because it makes a lot of get requests and doesn't handle failures,
it crashed often.
commit 5794e0403f2ad69c9ccd98733bceee908d1bf21e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Fri Nov 6 12:52:50 2020 +0100
Make the retry proxy work on all functions.
The metadata migration script kept crashing otherwise.
commit 9d1237b4eae1d64ca784e3ebb7f0f390e551b3a6
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Nov 5 15:28:49 2020 +0100
Set the value_sanitizer argument of get_journal_writer.
The next version of swh-journal will remove the default value.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1053/ for more details.
Comment Actions
Build is green
Patch application report for D4435 (id=15754)
Could not rebase; Attempt merge onto 24cdc85c15...
Updating 24cdc85c..c0a3d966 Fast-forward requirements-swh-journal.txt | 2 +- swh/storage/backfill.py | 10 +- swh/storage/migrate_extrinsic_metadata.py | 12 +- swh/storage/retry.py | 80 +--- swh/storage/tests/test_retry.py | 640 ------------------------------ swh/storage/writer.py | 21 +- 6 files changed, 53 insertions(+), 712 deletions(-)
Changes applied before test
commit c0a3d966faf45f4623c7e1bc03f5fa72024d22cf
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Fri Nov 6 12:54:14 2020 +0100
migrate_extrinsic_metadata: use the retry proxy
Because it makes a lot of get requests and doesn't handle failures,
it crashed often.
commit aded45b9b27d93df8195fde5f462250e2e84798b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Fri Nov 6 12:52:50 2020 +0100
Make the retry proxy work on all functions.
The metadata migration script kept crashing otherwise.
commit 2e7d489eb245d4c3b684360d6f70f061804f77a9
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Thu Nov 5 15:28:49 2020 +0100
Set the value_sanitizer argument of get_journal_writer.
The next version of swh-journal will remove the default value.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1061/ for more details.