Page MenuHomeSoftware Heritage

ContentIndexer: convert hash to bytes.
ClosedPublic

Authored by vlorentz on Thu, Dec 20, 6:35 PM.

Details

Summary

When received via the scheduler (which manipulates JSON), it's a string.

The RevisionIndexer uses the exact same conversion code.

Test Plan
docker-compose up
python3 -m swh.loader.git.loader --origin-url https://github.com/SoftwareHeritage/swh-storage.git
echo 'indexer_mimetype;oneshot;[["6dfe5dd2ab86d1ad3677285155027332fb35e9e5"]];{"policy_update": "update-dups"}' | python3 -m swh.scheduler.cli --cls remote --url http://localhost:5008/ task schedule /dev/stdin -c type -c policy -c args -c kwargs -d ';'

Diff Detail

Repository
rDCIDX Object indexer
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Thu, Dec 20, 6:35 PM
olasd accepted this revision.Fri, Dec 21, 3:37 PM
olasd added a subscriber: olasd.

I'm not too fond of silently accepting both data types (I'd rather have limitations of the scheduler handled on the task side rather than here) but I guess that's fine.

This also probably needs test coverage so we don't break it in the future.

swh/indexer/indexer.py
339

needs to be fixed

This revision is now accepted and ready to land.Fri, Dec 21, 3:37 PM
vlorentz added a comment.EditedFri, Dec 21, 3:46 PM
In D874#18797, @olasd wrote:

I'm not too fond of silently accepting both data types (I'd rather have limitations of the scheduler handled on the task side rather than here) but I guess that's fine.

I agree, but we already do it all over the indexers :/

This also probably needs test coverage so we don't break it in the future.

It's already tested, modulo D876

vlorentz updated this revision to Diff 2791.Fri, Dec 21, 3:48 PM
  • squash
This revision was automatically updated to reflect the committed changes.
Harbormaster failed remote builds in B3241: Diff 2791!