Page MenuHomeSoftware Heritage

pg: use 'on conflict do nothing' strategy for duplicate metadata rows.
ClosedPublic

Authored by vlorentz on Tue, Jun 30, 3:12 PM.

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Tue, Jun 30, 3:12 PM
vlorentz updated this revision to Diff 11993.Tue, Jun 30, 3:13 PM

explain the motivation in the commit message

Build is green

Patch application report for D3379 (id=11992)

Could not rebase; Attempt merge onto ffe6b9253e...

Updating ffe6b925..a7dc9347
Fast-forward
 swh/storage/cassandra/storage.py  |  66 +++++++++----------
 swh/storage/db.py                 |   4 +-
 swh/storage/in_memory.py          |  64 +++++++++----------
 swh/storage/interface.py          |  12 +++-
 swh/storage/storage.py            | 130 +++++++++++++++++++-------------------
 swh/storage/tests/test_storage.py |  19 ++++--
 6 files changed, 156 insertions(+), 139 deletions(-)
Changes applied before test
commit a7dc9347a3fed4036beb3e2b2940344acec0d105
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 15:11:49 2020 +0200

    pg: use 'on conflict do nothing' strategy for duplicate metadata rows.

commit 00f97f082e8e324a7a8f6d90b9bd1a1c0e708134
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 15:06:03 2020 +0200

    Document the behavior of adding a duplicate non-intrinsic object is unspecified.

commit 4c2bdadd353625d3b70285258745c207a318c95f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 14:56:20 2020 +0200

    Make the code location of metadata endpoints consistent across backends.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/397/ for more details.

ardumont accepted this revision.Tue, Jun 30, 3:19 PM
ardumont added a subscriber: ardumont.
ardumont added inline comments.
swh/storage/tests/test_storage.py
3487

maybe add something to tell this one is pg, this one is cass or something?

If i'm not mistaken, that'd be:

expected_results1,  # cass
expected_results2,  # pg
This revision is now accepted and ready to land.Tue, Jun 30, 3:19 PM

Build is green

Patch application report for D3379 (id=11993)

Could not rebase; Attempt merge onto ffe6b9253e...

Updating ffe6b925..6297cfe3
Fast-forward
 swh/storage/cassandra/storage.py  |  66 +++++++++----------
 swh/storage/db.py                 |   4 +-
 swh/storage/in_memory.py          |  64 +++++++++----------
 swh/storage/interface.py          |  12 +++-
 swh/storage/storage.py            | 130 +++++++++++++++++++-------------------
 swh/storage/tests/test_storage.py |  19 ++++--
 6 files changed, 156 insertions(+), 139 deletions(-)
Changes applied before test
commit 6297cfe3f62069d3f349f649d6cded73c54ebf82
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 15:11:49 2020 +0200

    pg: use 'on conflict do nothing' strategy for duplicate metadata rows.
    
    "updates are a problem for postgresql logical replication"

commit 00f97f082e8e324a7a8f6d90b9bd1a1c0e708134
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 15:06:03 2020 +0200

    Document the behavior of adding a duplicate non-intrinsic object is unspecified.

commit 4c2bdadd353625d3b70285258745c207a318c95f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 14:56:20 2020 +0200

    Make the code location of metadata endpoints consistent across backends.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/398/ for more details.

vlorentz updated this revision to Diff 11994.Tue, Jun 30, 3:26 PM

add comments

Build is green

Patch application report for D3379 (id=11994)

Could not rebase; Attempt merge onto ffe6b9253e...

Updating ffe6b925..953bd29a
Fast-forward
 swh/storage/cassandra/storage.py  |  66 +++++++++----------
 swh/storage/db.py                 |   4 +-
 swh/storage/in_memory.py          |  64 +++++++++----------
 swh/storage/interface.py          |  12 +++-
 swh/storage/storage.py            | 130 +++++++++++++++++++-------------------
 swh/storage/tests/test_storage.py |  19 ++++--
 6 files changed, 156 insertions(+), 139 deletions(-)
Changes applied before test
commit 953bd29ab04e4b567fbcafdd3ff02595128378b1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 15:11:49 2020 +0200

    pg: use 'on conflict do nothing' strategy for duplicate metadata rows.
    
    "updates are a problem for postgresql logical replication"

commit 00f97f082e8e324a7a8f6d90b9bd1a1c0e708134
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 15:06:03 2020 +0200

    Document the behavior of adding a duplicate non-intrinsic object is unspecified.

commit 4c2bdadd353625d3b70285258745c207a318c95f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jun 30 14:56:20 2020 +0200

    Make the code location of metadata endpoints consistent across backends.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/399/ for more details.