Page MenuHomeSoftware Heritage

Add a section in the README file on how to create tasks to start importing content asynchronously
ClosedPublic

Authored by douardda on Dec 18 2018, 6:34 PM.

Details

Summary

This requires to be able to easily run swh-scheduler commands, so the
first revision adds a 'swh-scheduler' command to the entrypoint, so one
can run things like:

docker-compose run swh-scheduler-api swh-scheduler task --list

This needs D854 to be applicable.

Diff Detail

Repository
rCDFD Dockerfiles for developers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.Dec 18 2018, 6:34 PM
olasd accepted this revision.Dec 18 2018, 6:53 PM
olasd added a subscriber: olasd.

I'm tentatively accepting this as better than the status quo, but while this probably does what it says on the tin, I fear that this task will have issues coming out the other end. :)

README.md
112–118 ↗(On Diff #2730)

Does this really work? The Debian lister needs to have stuff seeded in its database before being able to do anything useful (see swh.lister.debian.utils).

I suggest adding a gitlab lister task on a simple instance (I like 0xacab.org as it's fairly small, but it might not be PC enough :P)

157 ↗(On Diff #2730)

needs an update wrt D854.

dockerfiles/swh-scheduler-api/entrypoint.sh
28

the indentation seems to be broken (IME emacs likes mixing tabs and spaces in shell files)

This revision is now accepted and ready to land.Dec 18 2018, 6:53 PM
douardda marked 2 inline comments as done.Dec 19 2018, 3:14 PM
douardda added inline comments.
README.md
112–118 ↗(On Diff #2730)

The Debian lister needs to have stuff seeded in its database before being able to do anything useful

isn't what the ' --with-data' option of the lister's cli db initializer meant to do?

dockerfiles/swh-scheduler-api/entrypoint.sh
28

indeed

ardumont added inline comments.
README.md
112–118 ↗(On Diff #2730)

Yes, it is.

And yes, as per oral exchange from last week, that needs refactoring ;)

[1] https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/cli.py$56-72

douardda updated this revision to Diff 2739.Dec 19 2018, 4:54 PM

use a gitlab based example in readme and fix indentation in entrypoint

This revision was automatically updated to reflect the committed changes.