HomeSoftware Heritage

Refactor the generate_storage_from_git dataset creation tool

Description

Refactor the generate_storage_from_git dataset creation tool

use the newly introduced swh.journal StreamJournalWriter to generate the
msgpack-file with the content of the storage serialized, instead of
reimplementing a similar logic by hand.

Doing so, the msgpack file format is a little bit different, so current
datasets need to be regenerated.

From the 'consumer' side (the fill_storage fixture), simply use
swh.storage's process_replay_objects (once again, instead of reinventing
the wheel).

This is needed because we will need to handle much more model object types
for testing the origin-revision layer.

Details

Provenance
douarddaAuthored on Jun 17 2021, 7:53 PM
douarddaPushed on Jun 21 2021, 4:48 PM
Differential Revision
D5891: Refactor the generate_storage_from_git dataset creation tool
Parents
rDPROV8ff1ab5860a6: Improve .gitignore
Branches
Unknown
Tags
Unknown
Build Status
Buildable 22152
Build 34471: test-and-buildJenkins console · Jenkins