Page MenuHomeSoftware Heritage

in-memory storage: compute all counters
Closed, MigratedEdits Locked

Description

Summary:

"person", "origin_visit", and "skipped_content" objects are not counted by in_memory.Storage.stat_counters()


Context:

All *_add methods add full object content in a dedicated structure (eg. origin_add/origin_add_one adds an object in self._origins).
Additionally, most of them also add a "pointer" to that object in a common structure stored as self._objects, so they can be retrieved by object_find_by_sha1_git.

Not all of them do, because they manipulate objects that don't have a sha1, so we did not implement them because it wouldn't make sense.

However, self._objects is also used by stat_counters(); so counters about these objects are always 0.


What needs to be changed:

This issue resides entirely in swh-storage/swh/storage/in_memory.py.

To fix this issue, you need to add code to various *_add methods so they also add entries to self._objects, which will be counted by in_memory.Storage.stat_counters()

Event Timeline

vlorentz added a project: Easy hack.
vlorentz updated the task description. (Show Details)
zack renamed this task from in-memory storage: compute all counters. to in-memory storage: compute all counters.May 25 2019, 5:10 PM

I might be wrong but, I think it has been completed. Check out these :

But I see a lack of tests in test_in_memory.py. I can try doing that. What's your opinion?

vlorentz claimed this task.

ok, thanks. It's actually tested in test_stat_counters in swh-storage/swh/storage/tests/storage_tests.py, which is used to test all four classes.