Differential D7368

Add a journal checker
ClosedPublic
Actions

Authored by vlorentz on Mar 17 2022, 3:00 PM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Commits

rDSCRUB30bef11d276c: Add a journal checker

Diff Detail

Repository

rDSCRUB Datastore Scrubber

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 27526
Build 43077: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 43076: arc lint + arc unit

Event Timeline

vlorentz created this revision.Mar 17 2022, 3:00 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptMar 17 2022, 3:00 PM

Build is green

Patch application report for D7368 (id=26643)

Could not rebase; Attempt merge onto 5e030a65a9...

Updating 5e030a6..30bef11
Fast-forward
 README.rst                                    |   1 +
 conftest.py                                   |   1 +
 docs/README.rst                               |  34 +++++++
 requirements-swh.txt                          |   3 +
 requirements-test.txt                         |   3 +
 swh/scrubber/__init__.py                      |  23 +++++
 swh/scrubber/bar.py                           |   4 -
 swh/scrubber/check_journal.py                 |  61 ++++++++++++
 swh/scrubber/check_storage.py                 |  91 +++++++++++++++++
 swh/scrubber/cli.py                           | 137 +++++++++++++++++++++++++-
 swh/scrubber/db.py                            |  89 +++++++++++++++++
 swh/scrubber/sql/20-enums.sql                 |   1 +
 swh/scrubber/sql/30-schema.sql                |  28 ++++++
 swh/scrubber/sql/60-indexes.sql               |  13 +++
 swh/scrubber/tests/conftest.py                |  28 ++++++
 swh/scrubber/tests/test_cli.py                | 113 +++++++++++++++++++++
 swh/scrubber/tests/test_journal_kafka.py      | 120 ++++++++++++++++++++++
 swh/scrubber/tests/test_nothing.py            |   3 -
 swh/scrubber/tests/test_storage_postgresql.py | 105 ++++++++++++++++++++
 19 files changed, 849 insertions(+), 9 deletions(-)
 create mode 120000 README.rst
 create mode 100644 conftest.py
 delete mode 100644 swh/scrubber/bar.py
 create mode 100644 swh/scrubber/check_journal.py
 create mode 100644 swh/scrubber/check_storage.py
 create mode 100644 swh/scrubber/db.py
 create mode 100644 swh/scrubber/sql/20-enums.sql
 create mode 100644 swh/scrubber/sql/30-schema.sql
 create mode 100644 swh/scrubber/sql/60-indexes.sql
 create mode 100644 swh/scrubber/tests/conftest.py
 create mode 100644 swh/scrubber/tests/test_cli.py
 create mode 100644 swh/scrubber/tests/test_journal_kafka.py
 delete mode 100644 swh/scrubber/tests/test_nothing.py
 create mode 100644 swh/scrubber/tests/test_storage_postgresql.py

Changes applied before test

commit 30bef11d276ce509cf8fcee0489a5810690983ea
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Mar 17 14:48:49 2022 +0100

    Add a journal checker

commit be9a35c0c3977cb2b2f4ad2e9378d9cb394611f5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Mar 16 12:33:37 2022 +0100

    Initialize DB schema and postgresql storage checker

See https://jenkins.softwareheritage.org/job/DSCRUB/job/tests-on-diff/13/ for more details.

Harbormaster completed remote builds in B27526: Diff 26643.Mar 17 2022, 3:02 PM

vlorentz requested review of this revision.Mar 17 2022, 3:02 PM

vlorentz added a parent revision: D7360: Initialize DB schema and postgresql storage checker.Mar 17 2022, 3:14 PM

ardumont accepted this revision.Mar 21 2022, 10:55 AM

This revision is now accepted and ready to land.Mar 21 2022, 10:55 AM

Closed by commit rDSCRUB30bef11d276c: Add a journal checker (authored by vlorentz). · Explain WhyMar 22 2022, 11:20 AM

This revision was automatically updated to reflect the committed changes.

vlorentz added a commit: rDSCRUB30bef11d276c: Add a journal checker.