Page MenuHomeSoftware Heritage

Add a journal checker
ClosedPublic

Authored by vlorentz on Mar 17 2022, 3:00 PM.

Details

Diff Detail

Repository
rDSCRUB Datastore Scrubber
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 27526
Build 43077: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 43076: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D7368 (id=26643)

Could not rebase; Attempt merge onto 5e030a65a9...

Updating 5e030a6..30bef11
Fast-forward
 README.rst                                    |   1 +
 conftest.py                                   |   1 +
 docs/README.rst                               |  34 +++++++
 requirements-swh.txt                          |   3 +
 requirements-test.txt                         |   3 +
 swh/scrubber/__init__.py                      |  23 +++++
 swh/scrubber/bar.py                           |   4 -
 swh/scrubber/check_journal.py                 |  61 ++++++++++++
 swh/scrubber/check_storage.py                 |  91 +++++++++++++++++
 swh/scrubber/cli.py                           | 137 +++++++++++++++++++++++++-
 swh/scrubber/db.py                            |  89 +++++++++++++++++
 swh/scrubber/sql/20-enums.sql                 |   1 +
 swh/scrubber/sql/30-schema.sql                |  28 ++++++
 swh/scrubber/sql/60-indexes.sql               |  13 +++
 swh/scrubber/tests/conftest.py                |  28 ++++++
 swh/scrubber/tests/test_cli.py                | 113 +++++++++++++++++++++
 swh/scrubber/tests/test_journal_kafka.py      | 120 ++++++++++++++++++++++
 swh/scrubber/tests/test_nothing.py            |   3 -
 swh/scrubber/tests/test_storage_postgresql.py | 105 ++++++++++++++++++++
 19 files changed, 849 insertions(+), 9 deletions(-)
 create mode 120000 README.rst
 create mode 100644 conftest.py
 delete mode 100644 swh/scrubber/bar.py
 create mode 100644 swh/scrubber/check_journal.py
 create mode 100644 swh/scrubber/check_storage.py
 create mode 100644 swh/scrubber/db.py
 create mode 100644 swh/scrubber/sql/20-enums.sql
 create mode 100644 swh/scrubber/sql/30-schema.sql
 create mode 100644 swh/scrubber/sql/60-indexes.sql
 create mode 100644 swh/scrubber/tests/conftest.py
 create mode 100644 swh/scrubber/tests/test_cli.py
 create mode 100644 swh/scrubber/tests/test_journal_kafka.py
 delete mode 100644 swh/scrubber/tests/test_nothing.py
 create mode 100644 swh/scrubber/tests/test_storage_postgresql.py
Changes applied before test
commit 30bef11d276ce509cf8fcee0489a5810690983ea
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 17 14:48:49 2022 +0100

    Add a journal checker

commit be9a35c0c3977cb2b2f4ad2e9378d9cb394611f5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 16 12:33:37 2022 +0100

    Initialize DB schema and postgresql storage checker

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/13/ for more details.

This revision is now accepted and ready to land.Mar 21 2022, 10:55 AM
This revision was automatically updated to reflect the committed changes.