This is twice as fast, according to
https://forge.softwareheritage.org/T3577#72791
This is the same commit as D6495, rebased on D6885 instead of D6423.
Details
Details
- Reviewers
anlambert douardda - Group Reviewers
Reviewers - Maniphest Tasks
- T3577: Parallel loaders performances
- Commits
- rDSTOd5f1f0ec0554: cassandra: Rewrite content_missing to run queries concurrently.
Diff Detail
Diff Detail
- Repository
- rDSTO Storage manager
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 25954 Build 40563: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 40562: arc lint + arc unit
Event Timeline
Comment Actions
Build has FAILED
Patch application report for D6888 (id=24984)
Rebasing onto 4a24505049...
Current branch diff-target is up to date.
Changes applied before test
commit 2a38ccc4d43fefcc0652da780dd76ad403468323
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Mon Oct 18 13:25:20 2021 +0200
cassandra: Rewrite content_missing to run queries concurrently.
This is twice as fast, according to
https://forge.softwareheritage.org/T3577#72791Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1518/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1518/console
Comment Actions
Build is green
Patch application report for D6888 (id=24991)
Rebasing onto 4a24505049...
Current branch diff-target is up to date.
Changes applied before test
commit 55141ff2d57ca147efc2235eba2b006814c03817
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Mon Oct 18 13:25:20 2021 +0200
cassandra: Rewrite content_missing to run queries concurrently.
This is twice as fast, according to
https://forge.softwareheritage.org/T3577#72791See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1519/ for more details.
Comment Actions
fine for me (but plz give a bit more insight)
| swh/storage/cassandra/storage.py | ||
|---|---|---|
| 414 | would be nice to have a comment explaining why this more convoluted code is better (aka remind the reader the concurrency gained with the usage of content_find_many) | |
| 418 | not a big fan of the double for loop, but meh (alternative implem would probably be much worse) | |
Comment Actions
Build is green
Patch application report for D6888 (id=25080)
Rebasing onto 4a24505049...
Current branch diff-target is up to date.
Changes applied before test
commit d5f1f0ec055477461a000f7eeaece974fa1265b1
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Mon Oct 18 13:25:20 2021 +0200
cassandra: Rewrite content_missing to run queries concurrently.
This is twice as fast, according to
https://forge.softwareheritage.org/T3577#72791See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1522/ for more details.