see also T62

see also T22

This makes me think that we are now i/o bound on writes on our storage.

This task made good progress today. I spent a small while perusing our logging to understand the margins for performance.

• /revision/<SHA1_GIT>: show commit information
• /directory/<SHA1_GIT>: show directory information (including ls)
• /directory/<SHA1_GIT>/path/to/file-or-dir: ditto, but for dir pointed by path
• /content/[<HASH_ALGO>:]<HASH>: show content information
• /release/<SHA1_GIT>: show release information
• /person/<PERSON_ID>: show person information
• /origin/<ORIGIN_ID>: show origin information
• /project/<PROJECT_ID>: show project information
• /organization/<ORGANIZATION_ID>: show organization information
• /directory/<TIMESTAMP>/<ORIGIN>|/<BRANCH>|/path/to/file-or-dir : show directory information at timestamp/origin/branch
• /revision/<TIMESTAMP>/<ORIGIN>|/<BRANCH> : show revision information at origin/branch/timestamp
• /revision/<TIMESTAMP>/<ORIGIN>| : Show all branches of origin at a given timestamp
• /revision/<TIMESTAMP>/<ORIGIN>|/<BRANCH>| : Show all revisions (~git log) of origin and branch at a given timestamp

Done as of rDLDG69a5070

mv started on uffizi, in a screen session

Resolved as of rDSTO2b46e6941afe

As discussed on swh-private, this is no longer required now. We will reassess after having injected all the content we already have, selectively transfering only what we want/need.

• Done once with basic API
• Refactor to use an unified API call
• Keep up with latest change on swh-storage

done in rDSTObe3910ecff368967cbef7f803dbdf191c1510c3d (and subsequent fixups by olasd)

python3-swh.loader.git is installed and running on worker0{5..8}

gzip/checksumming restarted, after fixing the /etc/fstab mess on the machine

priority lowered as, for better or worse, we have already freed enough space on the machine for DB backups without having to transfer the data

the {exception:.../args:....} proposal + re-raising in the client looks good to me

We should standardize on :

• an HTTP error code (400 / Bad Request ?)
• a serialization format for the answer (JSON, probably serializing the error type and error args), e.g.

{
    'exception': e.__class__.__name__,
    'args': e.args,
}

and then deserialize that and raise in the client.

• code updated and deployed on worker0*
• all swh workers are currently running http://moma.internal.softwareheritage.org:5555/workers

As a first step to address this, in rDSTOTb841c72f6a240eac8131e40614c9ce20d75b1c96 I've checked into swh-storage-testdata several sample repositories, both as submodules and fast-export dumps.
Starting from those, we now need a Makefile target that batch import them into the SWH DB, and recreate the test database dumps.

now blocked due to the issues on sesi-pv-lc2 after the reboot :-/