Page MenuHomeSoftware Heritage
Differential D1992

writer: move the InMemoryJournalWriter from swh.storage in swh.journal
ClosedPublic
Actions

Authored by douardda on Sep 17 2019, 11:40 AM.

Details

Reviewers
vlorentz
ardumont
Group Reviewers
Reviewers
Commits
rDJNL6676f667a5d7: writer: rename writer.direct.DirectKafkaWriter as writer.kafka.
rDJNL9d505445b535: writer: move the InMemoryJournalWriter from swh.storage in swh.journal
Summary

and put both JournalWriter implementations in a swh.journal.writer package.
Also move the get_journal_writer() function from storage.

Keep a swh.journal.direct_writer module for bw compat.

This prevents a dependecy loop between swh.storage and swh.journal.

Diff Detail

Repository
rDJNL Journal infrastructure
Branch
master
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 7816
Build 11239: tox-on-jenkinsJenkins
Build 11238: arc lint + arc unit

Event Timeline

douardda created this revision.Sep 17 2019, 11:40 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptSep 17 2019, 11:40 AM
douardda added a child revision: D1993: journal_writer: use journal writer from swh.journal.Sep 17 2019, 11:42 AM
swh-public-ci added a comment.Sep 17 2019, 11:43 AM

Build is green
See https://jenkins.softwareheritage.org/job/DJNL/job/tox/251/ for more details.

Harbormaster completed remote builds in B7814: Diff 6714.Sep 17 2019, 11:43 AM
vlorentz added a subscriber: vlorentz.Sep 17 2019, 11:45 AM
vlorentz added inline comments.
swh/journal/writer/__init__.py
11

might as well rename it KafkaWriter while we're at it. "direct" was to be opposed to the pg listener and journal publisher, which were both dropped from the codebase, so it no long makes sense.

vlorentz accepted this revision.Sep 17 2019, 11:47 AM
This revision is now accepted and ready to land.Sep 17 2019, 11:47 AM
ardumont added a subscriber: ardumont.Sep 17 2019, 12:09 PM
ardumont added inline comments.
swh/journal/writer/__init__.py
11

Even JournalWriter directly?
This would be consistent with the in-memory version.

ardumont added inline comments.Sep 17 2019, 12:11 PM
swh/journal/writer/__init__.py
13

I know it's existing stuff we are moving around...
Might as well fix it now though 'Unknown journal writer class'...

ardumont requested changes to this revision.Sep 17 2019, 12:11 PM
This revision now requires changes to proceed.Sep 17 2019, 12:11 PM
douardda updated this revision to Diff 6716.Sep 17 2019, 1:18 PM

writer: rename writer.direct.DirectKafkaWriter as writer.kafka.KafkaJournalWriter

swh-public-ci added a comment.Sep 17 2019, 1:21 PM

Build is green
See https://jenkins.softwareheritage.org/job/DJNL/job/tox/252/ for more details.

Harbormaster completed remote builds in B7816: Diff 6716.Sep 17 2019, 1:21 PM
ardumont accepted this revision.Sep 17 2019, 1:28 PM
This revision is now accepted and ready to land.Sep 17 2019, 1:28 PM
Closed by commit rDJNL9d505445b535: writer: move the InMemoryJournalWriter from swh.storage in swh.journal (authored by douardda). · Explain WhySep 17 2019, 5:33 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathSize
swh/
journal/
4 lines
91 lines
tests/
test_kafka_writer.py
test_direct_writer.py
10 lines
4 lines
writer/
15 lines
30 lines
writer/
kafka.py
direct_writer.py
4 lines

Diff 6716

View Options

swh/journal/backfill.py

Loading...
View Options

swh/journal/direct_writer.py

Loading...
View Options

swh/journal/tests/test_direct_writer.py

Loading...
View Options

swh/journal/tests/test_kafka_writer.py

Loading...
View Options

swh/journal/tests/utils.py

Loading...
View Options

swh/journal/writer/__init__.py

Loading...
View Options

swh/journal/writer/inmemory.py

Loading...
View Options

swh/journal/writer/kafka.py

Loading...
About · Developer information · Legal