Page MenuHomeSoftware Heritage

storage_checker: Retry on postgresql errors from swh-storage
ClosedPublic

Authored by vlorentz on Dec 19 2022, 12:40 PM.

Details

Summary

This should avoid pointless reports to Sentry and Icinga on AdminShutdown
exceptions.

https://sentry.softwareheritage.org/share/issue/74f74880c807416988da16efa095be4a/

https://sentry.softwareheritage.org/share/issue/5796a55e3bb540b2ade33e4f52bc175b/

06:48:43 <+swhbot> icinga PROBLEM: service check_systemd on scrubber1.internal.softwareheritage.org is CRITICAL: SYSTEMD CRITICAL - swh-scrubber-checker-postgres@secondary-directory-0.service: failed, swh-scrubber-checker-postgres@secondary-directory-1.service: failed, swh-scrubber-checker-postgres@secondary-directory-2.service: failed, swh-scrubber-checker-postgres@secondary-directory-3.service: failed, swh-scrubber-checker-postgres@secondary-sn

Diff Detail

Repository
rDSCRUB Datastore Scrubber
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 33262
Build 52152: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 52151: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8968 (id=32307)

Rebasing onto 17cf8b6687...

Current branch diff-target is up to date.
Changes applied before test
commit e5403a82022a08f53503f08d7ce7946738e15905
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 12:39:39 2022 +0100

    storage_checker: Retry on postgresql errors from swh-storage
    
    This should avoid pointless reports to Sentry and Icinga on AdminShutdown
    exceptions.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/74/ for more details.

This revision is now accepted and ready to land.Dec 19 2022, 5:00 PM

Build is green

Patch application report for D8968 (id=32325)

Rebasing onto 2347dd723b...

Current branch diff-target is up to date.
Changes applied before test
commit a89cd6738094703dd5b78a7312a55bfc20cd11c3
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Mon Dec 19 12:39:39 2022 +0100

    storage_checker: Retry on postgresql errors from swh-storage
    
    This should avoid pointless reports to Sentry and Icinga on AdminShutdown
    exceptions.

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/75/ for more details.