Details
Diff Detail
- Repository
- rDSTO Storage manager
- Lint
Automatic diff as part of commit; lint not applicable. - Unit
Automatic diff as part of commit; unit tests not applicable.
Event Timeline
Build is green
Patch application report for D3008 (id=10676)
Could not rebase; Attempt merge onto ddac3d27e3...
Updating ddac3d2..93ca058 Fast-forward mypy.ini | 3 + swh/storage/backfill.py | 490 +++++++++++++++++++++++++++++ swh/storage/cli.py | 50 +++ swh/storage/tests/conftest.py | 45 ++- swh/storage/tests/test_api_client.py | 7 +- swh/storage/tests/test_api_client_dicts.py | 28 +- swh/storage/tests/test_backfill.py | 160 ++++++++++ swh/storage/tests/test_storage.py | 239 ++++++++------ 8 files changed, 900 insertions(+), 122 deletions(-) create mode 100644 swh/storage/backfill.py create mode 100644 swh/storage/tests/test_backfill.py
Changes applied before test
commit 93ca058fc487aa13f6c3e4895be1ea82aea2408a
Author: David Douard <david.douard@sdfa3.org>
Date: Thu Apr 9 16:56:44 2020 +0200
Copy the backfiller component from swh-journal
This componant makes more sense in the swh-storage package.
commit fd5ed546581506906dad640ba8c686a8ae754bbb
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Apr 7 16:17:39 2020 +0200
test: update storage tests to (future) swh.journal 0.0.30
which will handle swh.model objects everywhere instead of dicts.
Also add a BW compat version os the InMemoryJournalWriter so tests
will pass with current version of swh.journal (0.0.29).See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/86/ for more details.
Build is green
Patch application report for D3008 (id=10678)
Could not rebase; Attempt merge onto ddac3d27e3...
Updating ddac3d2..687fa2b Fast-forward mypy.ini | 3 + swh/storage/backfill.py | 490 +++++++++++++++++++++++++++++ swh/storage/cli.py | 50 +++ swh/storage/tests/conftest.py | 45 ++- swh/storage/tests/test_api_client.py | 7 +- swh/storage/tests/test_api_client_dicts.py | 28 +- swh/storage/tests/test_backfill.py | 160 ++++++++++ swh/storage/tests/test_storage.py | 239 ++++++++------ 8 files changed, 900 insertions(+), 122 deletions(-) create mode 100644 swh/storage/backfill.py create mode 100644 swh/storage/tests/test_backfill.py
Changes applied before test
commit 687fa2b3a4ba40ae58746b1504721c766dba5de9
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Apr 10 11:03:21 2020 +0200
cli: rename the command 'backfiller' as 'backfill'
for the sake of consistency.
commit 93ca058fc487aa13f6c3e4895be1ea82aea2408a
Author: David Douard <david.douard@sdfa3.org>
Date: Thu Apr 9 16:56:44 2020 +0200
Copy the backfiller component from swh-journal
This componant makes more sense in the swh-storage package.
commit fd5ed546581506906dad640ba8c686a8ae754bbb
Author: David Douard <david.douard@sdfa3.org>
Date: Tue Apr 7 16:17:39 2020 +0200
test: update storage tests to (future) swh.journal 0.0.30
which will handle swh.model objects everywhere instead of dicts.
Also add a BW compat version os the InMemoryJournalWriter so tests
will pass with current version of swh.journal (0.0.29).See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/87/ for more details.
Build is green
Patch application report for D3008 (id=10789)
Rebasing onto bca643acab...
Current branch diff-target is up to date.
Changes applied before test
commit b4784245fb4b0e1683b398dc0d7987a136b10763
Author: David Douard <david.douard@sdfa3.org>
Date: Thu Apr 9 16:56:44 2020 +0200
Copy the backfiller component from swh-journal
This componant makes more sense in the swh-storage package.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/101/ for more details.
Build is green
Patch application report for D3008 (id=10790)
Rebasing onto bca643acab...
Current branch diff-target is up to date.
Changes applied before test
commit 02f2ef2a00dd02156a8282aeedcff10a3b209e23
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Apr 10 11:03:21 2020 +0200
cli: rename the command 'backfiller' as 'backfill'
for the sake of consistency.
commit b4784245fb4b0e1683b398dc0d7987a136b10763
Author: David Douard <david.douard@sdfa3.org>
Date: Thu Apr 9 16:56:44 2020 +0200
Copy the backfiller component from swh-journal
This componant makes more sense in the swh-storage package.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/102/ for more details.
Build is green
Patch application report for D3008 (id=10846)
Rebasing onto fe56005555...
Current branch diff-target is up to date.
Changes applied before test
commit 698af8ca0028c5847326cf57f867608e533af9ed
Author: David Douard <david.douard@sdfa3.org>
Date: Fri Apr 10 11:03:21 2020 +0200
cli: rename the command 'backfiller' as 'backfill'
for the sake of consistency.
commit b2bba450aaa808406b024800dcd50b4778313903
Author: David Douard <david.douard@sdfa3.org>
Date: Thu Apr 9 16:56:44 2020 +0200
Copy the backfiller component from swh-journal
This componant makes more sense in the swh-storage package.See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/115/ for more details.
| swh/storage/backfill.py | ||
|---|---|---|
| 22 | Won't that pose a problem for module depending on the storage? it might be a transitional state though, i'm not up-to-date on where you are at with the transition. | |
| swh/storage/backfill.py | ||
|---|---|---|
| 22 | Not anymore than the other submodules of swh.storage already depending on swh.journal? | |
| swh/storage/backfill.py | ||
|---|---|---|
| 22 | Yes the idea is we already have the storage -> journal dependency but we should soon (!) have no mode journal -> storage dependency. I've let the swh.journal dependency in requirements-swh-journal.txt because technically, the journal is not mandatory for a simple storage. Not sure if we want to go more complex than that... | |
| swh/storage/backfill.py | ||
|---|---|---|
| 22 | I was thinking on those depending on the storage without the journal. Thanks. | |
| swh/storage/backfill.py | ||
|---|---|---|
| 22 | I'm waiting for the the current stack of diff to land in the storage as well as the to-be-sent swh-objstorage-replayer before submitting my pile of diffs that removes the backfiller and replayers from the journal. | |