Page MenuHomeSoftware Heritage

Ignore psqlrc when loading the rubygems database dump
ClosedPublic

Authored by olasd on Dec 5 2022, 1:52 PM.

Details

Summary

The SQL dump contains ownership instructions that can't be run if you
don't have the right users in your database clusters. When someone has a
psqlrc with ON_ERROR_STOP, this fails the load of the dump.

Use the opportunity to trigger an exception when psql returns a non-zero
exit code, rather than continue with an empty/inconsistent database.

Test Plan

tested on my laptop with a .psqlrc, the loading doesn't crash anymore

Diff Detail

Repository
rDLS Listers
Branch
detached-head
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 33075
Build 51844: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 51843: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D8914 (id=32122)

Rebasing onto f4aafe026b...

Current branch diff-target is up to date.
Changes applied before test
commit a66e24bfa2af7de8cacf4c4525f0eb5103aae0eb
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Mon Dec 5 13:43:32 2022 +0100

    Ignore psqlrc when loading the rubygems database dump
    
    The SQL dump contains ownership instructions that can't be run if you
    don't have the right users in your database clusters. When someone has a
    psqlrc with ON_ERROR_STOP, this fails the load of the dump.
    
    Use the opportunity to trigger an exception when psql returns a non-zero
    exit code, rather than continue with an empty/inconsistent database.

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/868/ for more details.

olasd requested review of this revision.Dec 5 2022, 1:58 PM

Thanks, I will try to update the swh/lister/rubygems/data/small_rubygems_dump.sh script to generate a postgres dump that can be loaded when ERROR_ON_STOP is set.

This revision is now accepted and ready to land.Dec 5 2022, 2:08 PM

Thanks, I will try to update the swh/lister/rubygems/data/small_rubygems_dump.sh script to generate a postgres dump that can be loaded when ERROR_ON_STOP is set.

There's not much point in doing that, the actual dump is generated with ownership instructions as well. I've wasted a good chunk of time trying to work around that but filtering their silly double-wrapped plain text SQL export is a PITA.

In D8914#231759, @olasd wrote:

Thanks, I will try to update the swh/lister/rubygems/data/small_rubygems_dump.sh script to generate a postgres dump that can be loaded when ERROR_ON_STOP is set.

There's not much point in doing that, the actual dump is generated with ownership instructions as well. I've wasted a good chunk of time trying to work around that but filtering their silly double-wrapped plain text SQL export is a PITA.

Ack, let's keep the lister in its current state then.