Page MenuHomeSoftware Heritage

Move Storage documentation and endpoint paths to a new StorageInterface class
ClosedPublic

Authored by vlorentz on Jan 24 2020, 5:53 PM.

Details

Summary

Documentation was duplicated between the in-mem and postgresql storage,
and one of them regularly goes out of date.
This deduplicates them both to a new class.

This new class is also the one declaring the API paths, as it did not
make sense to have this declaration on the postgresql storage.

Last but not least, this commit adds a test that checks backend classes
have all the functions, and they have exactly the same signature as the
interface. This will catch stupid bugs before production, eg. if an argument
does not have the same name in all classes.

Depends on D2583, D2584, D2586.

Diff Detail

Repository
rDSTO Storage manager
Branch
storage-interface
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 10285
Build 15272: tox-on-jenkinsJenkins
Build 15271: arc lint + arc unit

Event Timeline

Remove forgotten docstrings

I started the review and asked for docstring updates.
I was gonna ask to add types as well.
But i stopped.

I guess we'd be better off go and do that in another dedicated and less big diff.

swh/storage/interface.py
15

what's the difference with pass keyword?

23

Remove that note and mark the method idempotent in the docstring.

Add content blobs to the storage idempotently or something better if you have that in store.

44

remove that note as it's specific to the db implementation.
Maybe move it there (don't know if we can do so now without overwriting all docstrings).

84

without objstorage insertion

swh/storage/tests/test_storage.py
105

thx for the comment!

107

why is this one aside?
isn't it some left-overs from a tryout?

This revision is now accepted and ready to land.Jan 27 2020, 9:47 AM
swh/storage/tests/test_storage.py
100

here.

swh/storage/tests/test_storage.py
107

I'm using a bit of magic, so I added an assertion to check the magic doesn't remove all the declared methods from dir().

apply comment on content_add.

swh/storage/interface.py
15

Semantically, not much.

But it's what mypy docs use to define protocols: https://mypy.readthedocs.io/en/stable/protocols.html#simple-user-defined-protocols

23

why? It is true of any backend.

44

indeed

84

what's the difference? and I did not write this btw, it's just copy-pasted

  • rebase
  • fix some methods missing from copy-paste