Rebase and add a test, as requested.

rebase

rebase

get rid of the attrib_typecheck() decorator to prevent the need for a mypy plugin

In D2769#67614, @vlorentz wrote:

In D2769#67431, @douardda wrote:

The annotation part should be done on the whole module and, most importantly, in a dedicated revision.

The actual fix consist in a single 'if snapshot:' but this not what the diffstat "shows".

I only added annotations on code I changed.

In D2769#68176, @ardumont wrote:

In D2769#67614, @vlorentz wrote:

In D2769#67431, @douardda wrote:

The annotation part should be done on the whole module and, most importantly, in a dedicated revision.

The actual fix consist in a single 'if snapshot:' but this not what the diffstat "shows".

I only added annotations on code I changed.

Wait, i found those annotation change on the code changed reasonable.
Against spend an insane amount of time typing all the module in one go (which must be a pita).
I prefer the incremental approach.

@douardda So what's wrong here?

The fact that val did not say he added types it in the description or that he did add the types?

I kinda do the same.
I increase the types of the code when i'm incrementally changing it.
I'd like to know if that's wrong.

Also BaseModel.from_dict is currently pretty inconsistent: sometimes it will take care of instanciating model entities for attributes (e.g. for TimestampWithTimezone.timestamp), sometimes not (e.g. SkippedContent.origin)

rebased

apply vlorentz' comments

add missing mypy deps in requirements-test

add support for the 'validator' argument in attrib_typecheck

rebase + add missing plugin file

replace bhex() by _x() and other stuff reported by olasd

The annotation part should be done on the whole module and, most importantly, in a dedicated revision.

In D2771#66331, @vlorentz wrote:

I really think we should either have it for all object types or none at all.

In D2814#67371, @ardumont wrote:

What about adding tests on this or do you rely on BaseModel's?

In D2803#67209, @olasd wrote:

In D2803#67208, @douardda wrote:

In D2803#67024, @olasd wrote:

My main doubt was whether we stopped explicitly converting model objects to dicts altogether (going through the swh.core model serializer instead). But even in that case contents will still be deserializable (as Content.from_dict(d) still works even when d['data'] is None).

What swh.core model serializer do you refer to? The ones in swh.core.api?

Yes. And now that you've pointed it out, I've remembered that it's the swh.storage RPC layer that adds a hook to support model objects.

In D2803#67024, @olasd wrote:

My main doubt was whether we stopped explicitly converting model objects to dicts altogether (going through the swh.core model serializer instead). But even in that case contents will still be deserializable (as Content.from_dict(d) still works even when d['data'] is None).

remove extra parameter 'anon' mistakenly included in the diff

In D2776#66377, @olasd wrote:

This looks sound but the tests are hanging on the initialization of the postgresql database now... (at least on jenkins)

ok (besides my remark).

This is nice.

You should give a hint in your commit message on why you do this refactoring.

okay-ish but lifecycle of ES related services/objects is unclear to me.

Thanks for the contribution.
You must however ensure tests pass ok before we can accept it. Note that the tests you modify (in tests/test_storage.py) are executed by all the storage backends (postgres, cassandra and the in_memory one you really are targeting here). So make sure they are still OK with all the backends.