Page MenuHomeSoftware Heritage

cassandra: Add a test of a 'complex' migration, with a PK update
ClosedPublic

Authored by vlorentz on Fri, Apr 23, 10:43 AM.

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D5584 (id=19953)

Could not rebase; Attempt merge onto eb8c147a54...

Updating eb8c147a..91112860
Fast-forward
 swh/storage/cassandra/model.py                |   7 +-
 swh/storage/cassandra/storage.py              | 145 +++++++----
 swh/storage/in_memory.py                      |   1 +
 swh/storage/tests/test_cassandra_migration.py | 341 ++++++++++++++++++++++++++
 4 files changed, 441 insertions(+), 53 deletions(-)
 create mode 100644 swh/storage/tests/test_cassandra_migration.py
Changes applied before test
commit 91112860b9fd144a66a25960387d18394cda162b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 23 10:43:27 2021 +0200

    cassandra: Add a test of a 'complex' migration, with a PK update

commit 879b240c123dc47afffc6bcb3840be32d6bf1eab
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Apr 22 20:27:33 2021 +0200

    cassandra: Add 'check_missing' option, to allow updating objects
    
    as part of a migration.
    
    Also write a first test that simulates how a simple migration would go.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1268/ for more details.

douardda added a subscriber: douardda.

I would not say all this is crystal clear to me, but overall looks fine.

This revision is now accepted and ready to land.Tue, Apr 27, 10:34 AM

rebase + rename 'check_missing' to 'not allow_overwrite'

Build has FAILED

Patch application report for D5584 (id=20127)

Could not rebase; Attempt merge onto 98804f9e12...

Merge made by the 'recursive' strategy.
 swh/storage/cassandra/model.py                |   7 +-
 swh/storage/cassandra/storage.py              | 148 +++++++----
 swh/storage/in_memory.py                      |   1 +
 swh/storage/tests/test_cassandra_migration.py | 341 ++++++++++++++++++++++++++
 4 files changed, 444 insertions(+), 53 deletions(-)
 create mode 100644 swh/storage/tests/test_cassandra_migration.py
Changes applied before test
commit 7a608c3f3cf7a215d97aae04ddbe5a5d69921d98
Merge: 98804f9e fe64cab5
Author: Jenkins user <jenkins@localhost>
Date:   Thu Apr 29 08:23:26 2021 +0000

    Merge branch 'diff-target' into HEAD

commit fe64cab5d93a861290db8e993ede93c34fb41360
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 23 10:43:27 2021 +0200

    cassandra: Add a test of a 'complex' migration, with a PK update

commit 25bc523062eba8b5d75f9dc96746cd339e29ce6e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Apr 22 20:27:33 2021 +0200

    cassandra: Add 'check_missing' option, to allow updating objects
    
    as part of a migration.
    
    Also write a first test that simulates how a simple migration would go.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1280/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1280/console

Build has FAILED

Patch application report for D5584 (id=20130)

Could not rebase; Attempt merge onto 98804f9e12...

Merge made by the 'recursive' strategy.
 swh/storage/cassandra/model.py                |   7 +-
 swh/storage/cassandra/storage.py              | 148 +++++++----
 swh/storage/in_memory.py                      |   1 +
 swh/storage/tests/test_cassandra_migration.py | 341 ++++++++++++++++++++++++++
 4 files changed, 444 insertions(+), 53 deletions(-)
 create mode 100644 swh/storage/tests/test_cassandra_migration.py
Changes applied before test
commit c644e62433eae3cdfc8e395b95e9b2d352f8f5c4
Merge: 98804f9e ef1a018a
Author: Jenkins user <jenkins@localhost>
Date:   Thu Apr 29 09:02:04 2021 +0000

    Merge branch 'diff-target' into HEAD

commit ef1a018adce2e3e9ea72109f7bcc41b4a38fb2bf
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 23 10:43:27 2021 +0200

    cassandra: Add a test of a 'complex' migration, with a PK update

commit 6edad20297793e2fed9fbd3f8042f4a7741d90ba
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Apr 22 20:27:33 2021 +0200

    cassandra: Add 'check_missing' option, to allow updating objects
    
    as part of a migration.
    
    Also write a first test that simulates how a simple migration would go.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1282/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1282/console

Build has FAILED

Patch application report for D5584 (id=20139)

Could not rebase; Attempt merge onto 98804f9e12...

Merge made by the 'recursive' strategy.
 swh/storage/cassandra/model.py                |   7 +-
 swh/storage/cassandra/storage.py              | 150 +++++++----
 swh/storage/in_memory.py                      |   1 +
 swh/storage/tests/test_cassandra.py           | 109 +++++++-
 swh/storage/tests/test_cassandra_migration.py | 341 ++++++++++++++++++++++++++
 5 files changed, 553 insertions(+), 55 deletions(-)
 create mode 100644 swh/storage/tests/test_cassandra_migration.py
Changes applied before test
commit fc1f64022568e41fe8cc853c8f6ddce562b30f5f
Merge: 98804f9e f189ac34
Author: Jenkins user <jenkins@localhost>
Date:   Thu Apr 29 10:31:41 2021 +0000

    Merge branch 'diff-target' into HEAD

commit f189ac34b0367f256eec323bb52166660f390a4a
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 23 10:43:27 2021 +0200

    cassandra: Add a test of a 'complex' migration, with a PK update

commit 7567146f7b712b2c1b27f61682e391f120f276d0
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Apr 22 20:27:33 2021 +0200

    cassandra: Add 'check_missing' option, to allow updating objects
    
    as part of a migration.
    
    Also write a first test that simulates how a simple migration would go.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1284/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1284/console

Build is green

Patch application report for D5584 (id=20145)

Could not rebase; Attempt merge onto 98804f9e12...

Merge made by the 'recursive' strategy.
 swh/storage/cassandra/model.py                |   7 +-
 swh/storage/cassandra/storage.py              | 152 ++++++++----
 swh/storage/in_memory.py                      |   1 +
 swh/storage/tests/test_cassandra.py           | 109 +++++++-
 swh/storage/tests/test_cassandra_migration.py | 341 ++++++++++++++++++++++++++
 5 files changed, 555 insertions(+), 55 deletions(-)
 create mode 100644 swh/storage/tests/test_cassandra_migration.py
Changes applied before test
commit 84bf14b27c8118ac2da240a2a03ea3d35e5d8d9e
Merge: 98804f9e df4bff6f
Author: Jenkins user <jenkins@localhost>
Date:   Thu Apr 29 12:23:00 2021 +0000

    Merge branch 'diff-target' into HEAD

commit df4bff6f5614a34a4936326a0c0ca5b93383a55f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 23 10:43:27 2021 +0200

    cassandra: Add a test of a 'complex' migration, with a PK update

commit 5a0915c569fd9db649eaaea14e02a32c8aec1909
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Apr 22 20:27:33 2021 +0200

    cassandra: Add 'check_missing' option, to allow updating objects
    
    as part of a migration.
    
    Also write a first test that simulates how a simple migration would go.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1288/ for more details.

Build is green

Patch application report for D5584 (id=20231)

Could not rebase; Attempt merge onto 92d551a4a5...

Updating 92d551a4..051b7715
Fast-forward
 swh/storage/cassandra/model.py                |   7 +-
 swh/storage/cassandra/storage.py              | 152 ++++++++----
 swh/storage/in_memory.py                      |   1 +
 swh/storage/tests/test_cassandra.py           | 109 +++++++-
 swh/storage/tests/test_cassandra_migration.py | 341 ++++++++++++++++++++++++++
 5 files changed, 555 insertions(+), 55 deletions(-)
 create mode 100644 swh/storage/tests/test_cassandra_migration.py
Changes applied before test
commit 051b771523567da7fc70b7937eb3d1a6ec27a85d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Fri Apr 23 10:43:27 2021 +0200

    cassandra: Add a test of a 'complex' migration, with a PK update

commit f23346144dd3032df0664b3a49b8fe254aed217f
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Apr 22 20:27:33 2021 +0200

    cassandra: Add 'check_missing' option, to allow updating objects
    
    as part of a migration.
    
    Also write a first test that simulates how a simple migration would go.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1298/ for more details.