Page MenuHomeSoftware Heritage

origin_visit_upsert: Use OriginVisit objects as input
ClosedPublic

Authored by ardumont on Mar 11 2020, 5:38 PM.

Details

Summary

This aligns with other _add endpoints [1]

Impacts:

  • D2826: swh-journal (replayer)

Related to D2812#67298

[1] except for origin_visit_add which is next (D2820)

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont edited the summary of this revision. (Show Details)
vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/storage/cassandra/storage.py
829

This shouldn't type-check.

Call json.dumps in CqlRunner.origin_visit_upsert, like CqlRunner.origin_visit_update does.

swh/storage/db.py
382

Do we want swh-model objects in db.py?

This revision now requires changes to proceed.Mar 12 2020, 2:27 PM
swh/storage/db.py
382

why not?

swh/storage/db.py
382

Because we didn't do it so far, so this introduces an inconsistency.

swh/storage/db.py
382

but we are trying to fix that inconsistency everywhere by using model object...
I'm trying to get there...

swh/storage/cassandra/storage.py
829

ok, will do.

swh/storage/db.py
382

also you typed cql with base model object
This cql (cassandra) module is the equivalent of the db (pg) here...
So that's the consistent way forward.

swh/storage/db.py
382

Sure, but I would rather this single function is consistent with the rest of its module, than with a completely different module.

But if we want to make db.py use model objects, then sure

swh/storage/cassandra/storage.py
829

This shouldn't type-check.

metadata = attr.ib(type=Optional[Dict[str, object]],
                   default=None)

I see now. json.dumps could return a string, list or Dict...

Well, that's actually a field which is not used...

swh/storage/cassandra/storage.py
829

json.dumps could return a string, list or Dict...

no, just a string...
ok i see (pfff, needs some nap)

swh/storage/db.py
382

Well consistency is the goal.
But i'm doing that incrementally as i go along.

This revision is now accepted and ready to land.Mar 13 2020, 11:31 AM