Page MenuHomeSoftware Heritage

storage: Add not null constraints to metadata_authority/origin_metadata tables
ClosedPublic

Authored by ardumont on Wed, May 20, 9:21 AM.

Details

Summary

There were discrepancies between the latest 149.sql and the latest schema.sql.

As I was fighting the replication at the time already, I prefered aligning
both. Out of 149, I understood we wanted to make not null constraints on the
following tables, so here goes that diff.

This commits fixes that, migrating first the data not respecting those
constraints.

Related to D2988
Related to T2075

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Wed, May 20, 9:21 AM

Build has FAILED

Patch application report for D3170 (id=11254)

Rebasing onto c69d1001fd...

Current branch diff-target is up to date.
Changes applied before test
commit f82523e14fe7f06df0fb1d4ba74ea3bef3daf6ba
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 19 22:29:14 2020 +0200

    Add not null constraints to metadata_authority/origin_metadata
    
    Related to D2988
    Related to T2075

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/185/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/185/console

Well, well, now i need to fix those.

ardumont edited the summary of this revision. (Show Details)Wed, May 20, 10:39 AM
ardumont updated this revision to Diff 11257.Wed, May 20, 10:55 AM
ardumont edited the summary of this revision. (Show Details)

Fix typos in sql model queries ¯\_(ツ)_/¯

Also rework commit message

Build is green

Patch application report for D3170 (id=11257)

Rebasing onto c69d1001fd...

Current branch diff-target is up to date.
Changes applied before test
commit 96354ee3b526c0c39489d9113164216ef6cb15ee
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 19 22:29:14 2020 +0200

    Add not null constraints to metadata_authority/origin_metadata
    
    Out of 149, I understood we wanted to make not null constraints on the
    following tables, so here it goes.
    
    This commits fixes that, migrating first the data not respecting those
    constraints.
    
    Related to D2988
    Related to T2075

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/186/ for more details.

vlorentz requested changes to this revision.Wed, May 20, 11:48 AM
vlorentz added inline comments.
swh/storage/sql/30-swh-schema.sql
420
This revision now requires changes to proceed.Wed, May 20, 11:48 AM
ardumont added inline comments.Wed, May 20, 11:52 AM
swh/storage/sql/30-swh-schema.sql
420

Ok, removing the default.

We have some which are None already though.
Is it fine to default it to {} as the migration current does?

softwareheritage=> select * from metadata_authority where metadata is null;
 id  |      type      |           url            | metadata
-----+----------------+--------------------------+----------
 206 | deposit_client | https://www.ipol.im/     |
 250 | deposit_client | https://doi.org/10.5201/ |
ardumont updated this revision to Diff 11263.Wed, May 20, 12:09 PM

Adapt according to review

vlorentz accepted this revision.Wed, May 20, 12:14 PM
This revision is now accepted and ready to land.Wed, May 20, 12:14 PM

Build is green

Patch application report for D3170 (id=11263)

Rebasing onto c69d1001fd...

Current branch diff-target is up to date.
Changes applied before test
commit f1b51a7a2bdd71c1676b8bcee5ac28816e369dcb
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue May 19 22:29:14 2020 +0200

    Add not null constraints to metadata_authority/origin_metadata
    
    Out of 149, I understood we wanted to make not null constraints on the
    following tables, so here it goes.
    
    This commits fixes that, migrating first the data not respecting those
    constraints.
    
    Related to D2988
    Related to T2075

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/187/ for more details.