Page MenuHomeSoftware Heritage
Differential D3814

Add a check_dbversion() method to the Db class
ClosedPublic
Actions

Authored by douardda on Aug 20 2020, 12:10 PM.

Details

Reviewers
vlorentz
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T2525: Prevent the storage server from starting in case of db version mismatch
Commits
rDSTOc16ff506ad52: Add a check_dbversion() method to the Db class
rDSTO629d2d10ee97: Fix pytest_plugin's database janitor: do not truncate the dbversion table
Summary

This method compares the version stored in the database in the dbversion
table with the currently declared version.

This current version is declared as a simple current_version attribute
on the Db class. It must be updated jointly in 30-swh-schema.sql when
modifying the db schema. Note that if this is forgotten, the added test
test_dbversion should fail.

Related to T2525.

Also:

  • Code cleanup: fix import statements swh.storage.common content has been moved to swh.core.db.common a while ago.
  • Fix pytest_plugin's database janitor: do not truncate the dbversion table

Diff Detail

Event Timeline

douardda created this revision.Aug 20 2020, 12:10 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptAug 20 2020, 12:10 PM
swh-public-ci added a comment.Aug 20 2020, 12:15 PM

Build has FAILED

Patch application report for D3814 (id=13428)

Rebasing onto 6a53cb3ef3...

Current branch diff-target is up to date.
Changes applied before test
commit 70294c0a364d25d16ea2230a71bd242fd38a81c8
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 12:00:37 2020 +0200

    Add a check_dbversion() method to the Db class
    
    This method compares the version stored in the database in the `dbversion`
    table with the currently declared version.
    
    This current version is declared as a simple `current_version` attribute
    on the Db class. **It must be updated jointly in 30-swh-schema.sql** when
    modifying the db schema. Note that if this is forgotten, the added test
    `test_dbversion` should fail.
    
    Related to T2525.

commit 418a8f0f9548f7d735473d0d893b653b10758a34
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:55:56 2020 +0200

    Fix pytest_plugin's database janitor: do not truncate the dbversion table

commit 79428520e23aa00e20960071c342b8279f406f22
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:57:22 2020 +0200

    Code cleanup: fix import statements
    
    `swh.storage.common` content has been moved to `swh.core.db.common` a while ago.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/830/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/830/console

Harbormaster failed remote builds in B14692: Diff 13428!Aug 20 2020, 12:15 PM
douardda updated this revision to Diff 13430.EditedAug 20 2020, 12:21 PM

as said in the commit message, It must be updated jointly in 30-swh-schema.sql
(including after a rebase...)

swh-public-ci added a comment.Aug 20 2020, 12:25 PM

Build is green

Patch application report for D3814 (id=13430)

Rebasing onto 6a53cb3ef3...

Current branch diff-target is up to date.
Changes applied before test
commit 9e29879a30067bcabdd97ec721db5d627f658f14
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 12:00:37 2020 +0200

    Add a check_dbversion() method to the Db class
    
    This method compares the version stored in the database in the `dbversion`
    table with the currently declared version.
    
    This current version is declared as a simple `current_version` attribute
    on the Db class. **It must be updated jointly in 30-swh-schema.sql** when
    modifying the db schema. Note that if this is forgotten, the added test
    `test_dbversion` should fail.
    
    Related to T2525.

commit 418a8f0f9548f7d735473d0d893b653b10758a34
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:55:56 2020 +0200

    Fix pytest_plugin's database janitor: do not truncate the dbversion table

commit 79428520e23aa00e20960071c342b8279f406f22
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:57:22 2020 +0200

    Code cleanup: fix import statements
    
    `swh.storage.common` content has been moved to `swh.core.db.common` a while ago.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/831/ for more details.

Harbormaster completed remote builds in B14694: Diff 13430.Aug 20 2020, 12:25 PM
douardda updated this revision to Diff 13451.Aug 20 2020, 4:33 PM

rebase

swh-public-ci added a comment.Aug 20 2020, 4:38 PM

Build is green

Patch application report for D3814 (id=13451)

Rebasing onto 3ac332ecbf...

Current branch diff-target is up to date.
Changes applied before test
commit fcf09599f3462050a592b4b93c4a60a7eb5ab0cd
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 12:00:37 2020 +0200

    Add a check_dbversion() method to the Db class
    
    This method compares the version stored in the database in the `dbversion`
    table with the currently declared version.
    
    This current version is declared as a simple `current_version` attribute
    on the Db class. **It must be updated jointly in 30-swh-schema.sql** when
    modifying the db schema. Note that if this is forgotten, the added test
    `test_dbversion` should fail.
    
    Related to T2525.

commit 2f0dff41a9237c9e01413df37fbeef04697166f2
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:55:56 2020 +0200

    Fix pytest_plugin's database janitor: do not truncate the dbversion table

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/838/ for more details.

Harbormaster completed remote builds in B14714: Diff 13451.Aug 20 2020, 4:38 PM
douardda updated this revision to Diff 13461.Aug 20 2020, 5:43 PM

rebase

swh-public-ci added a comment.Aug 20 2020, 5:48 PM

Build is green

Patch application report for D3814 (id=13461)

Rebasing onto d1f19e97c3...

Current branch diff-target is up to date.
Changes applied before test
commit cba6959b163ce0711b3b490201dd6f414843396e
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 12:00:37 2020 +0200

    Add a check_dbversion() method to the Db class
    
    This method compares the version stored in the database in the `dbversion`
    table with the currently declared version.
    
    This current version is declared as a simple `current_version` attribute
    on the Db class. **It must be updated jointly in 30-swh-schema.sql** when
    modifying the db schema. Note that if this is forgotten, the added test
    `test_dbversion` should fail.
    
    Related to T2525.

commit 346ad9c4197e57d377fed351b2353e14af486995
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:55:56 2020 +0200

    Fix pytest_plugin's database janitor: do not truncate the dbversion table

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/842/ for more details.

Harbormaster completed remote builds in B14724: Diff 13461.Aug 20 2020, 5:48 PM
vlorentz accepted this revision.Aug 21 2020, 11:47 PM
This revision is now accepted and ready to land.Aug 21 2020, 11:47 PM
douardda added a child revision: D3831: Check for db version mismatch in PgStorage.check_config().Aug 24 2020, 2:50 PM
ardumont added a subscriber: ardumont.Aug 24 2020, 3:36 PM
ardumont added inline comments.
swh/storage/postgresql/db.py
1337

black did play some trick ;)
also the f is no longer needed (this should say something, shouldn't it?)

ardumont accepted this revision.Aug 24 2020, 3:37 PM

a string which got realigned can be merged into 1
but otherwise, fine ;)

douardda added inline comments.Aug 24 2020, 4:36 PM
swh/storage/postgresql/db.py
1337

:-)

douardda updated this revision to Diff 13489.Aug 24 2020, 4:40 PM

fix failed black magic as ween by ardumont

swh-public-ci added a comment.Aug 24 2020, 4:44 PM

Build is green

Patch application report for D3814 (id=13489)

Rebasing onto f570f9336c...

First, rewinding head to replay your work on top of it...
Applying: Fix pytest_plugin's database janitor: do not truncate the dbversion table
Applying: Add a check_dbversion() method to the Db class
Changes applied before test
commit e9347f25df37ca7a8a21a44761b7496d4b1e5b1b
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 12:00:37 2020 +0200

    Add a check_dbversion() method to the Db class
    
    This method compares the version stored in the database in the `dbversion`
    table with the currently declared version.
    
    This current version is declared as a simple `current_version` attribute
    on the Db class. **It must be updated jointly in 30-swh-schema.sql** when
    modifying the db schema. Note that if this is forgotten, the added test
    `test_dbversion` should fail.
    
    Related to T2525.

commit 652e03fc16de599921597910b2b504a371aaa0d9
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:55:56 2020 +0200

    Fix pytest_plugin's database janitor: do not truncate the dbversion table

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/851/ for more details.

Harbormaster completed remote builds in B14749: Diff 13489.Aug 24 2020, 4:44 PM
douardda updated this revision to Diff 13493.Aug 24 2020, 4:48 PM

pick the correct commit to be stashed...

douardda updated this revision to Diff 13498.Aug 24 2020, 4:59 PM

rebase

swh-public-ci added a comment.Aug 24 2020, 5:03 PM

Build is green

Patch application report for D3814 (id=13493)

Rebasing onto f570f9336c...

First, rewinding head to replay your work on top of it...
Applying: Fix pytest_plugin's database janitor: do not truncate the dbversion table
Applying: Add a check_dbversion() method to the Db class
Changes applied before test
commit e164157f7e89c47b018d5143c9ff2c3b2cceb446
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 12:00:37 2020 +0200

    Add a check_dbversion() method to the Db class
    
    This method compares the version stored in the database in the `dbversion`
    table with the currently declared version.
    
    This current version is declared as a simple `current_version` attribute
    on the Db class. **It must be updated jointly in 30-swh-schema.sql** when
    modifying the db schema. Note that if this is forgotten, the added test
    `test_dbversion` should fail.
    
    Related to T2525.

commit 77986148fb6d8caf6671f950ff443db74c3ea357
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:55:56 2020 +0200

    Fix pytest_plugin's database janitor: do not truncate the dbversion table

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/855/ for more details.

Harbormaster completed remote builds in B14753: Diff 13493.Aug 24 2020, 5:03 PM
swh-public-ci added a comment.Aug 24 2020, 5:20 PM

Build is green

Patch application report for D3814 (id=13498)

Rebasing onto f570f9336c...

Current branch diff-target is up to date.
Changes applied before test
commit c16ff506ad52abb926c656cb6cd0872ebcf9bf58
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 12:00:37 2020 +0200

    Add a check_dbversion() method to the Db class
    
    This method compares the version stored in the database in the `dbversion`
    table with the currently declared version.
    
    This current version is declared as a simple `current_version` attribute
    on the Db class. **It must be updated jointly in 30-swh-schema.sql** when
    modifying the db schema. Note that if this is forgotten, the added test
    `test_dbversion` should fail.
    
    Related to T2525.

commit 629d2d10ee97ddca057858d5a2fe2d6b52f6edd7
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Aug 20 11:55:56 2020 +0200

    Fix pytest_plugin's database janitor: do not truncate the dbversion table

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/859/ for more details.

Harbormaster completed remote builds in B14758: Diff 13498.Aug 24 2020, 5:20 PM
Closed by commit rDSTO629d2d10ee97: Fix pytest_plugin's database janitor: do not truncate the dbversion table (authored by douardda). · Explain WhyAug 24 2020, 5:24 PM
This revision was automatically updated to reflect the committed changes.
douardda added a commit: rDSTO629d2d10ee97: Fix pytest_plugin's database janitor: do not truncate the dbversion table.
douardda added a commit: rDSTOc16ff506ad52: Add a check_dbversion() method to the Db class.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
storage/
postgresql/
12 lines97%
Loading...
2 lines98%
Loading...
tests/
10 lines93%
Loading...
10 lines97%
Loading...

Diff 13489

View Options

swh/storage/postgresql/db.py

Loading...
View Options

swh/storage/pytest_plugin.py

Loading...
View Options

swh/storage/tests/test_cli.py

Loading...
View Options

swh/storage/tests/test_postgresql.py

Loading...
About · Developer information · Legal