Changeset View
Changeset View
Standalone View
Standalone View
README.md
Show First 20 Lines • Show All 98 Lines • ▼ Show 20 Lines | |||||
### Inserting a new lister task | ### Inserting a new lister task | ||||
To list the content of a source code provider like github or the Debian | To list the content of a source code provider like github or the Debian | ||||
distribution, you may add a new task for this. | distribution, you may add a new task for this. | ||||
This task should then spawn a series of loader tasks. | This task should then spawn a series of loader tasks. | ||||
For example, to add a recurring task that will scrape and maintain updated | For example, to add a (on shot) task that will list git repos on the 0xacab.org | ||||
vlorentz: "one shot" | |||||
the Debian stretch distribution, one can do (from this git repository): | gitlab instance, one can do (from this git repository): | ||||
``` | ``` | ||||
$ docker-compose run swh-scheduler-api \ | $ docker-compose run swh-scheduler-api \ | ||||
swh-scheduler -c remote -u http://swh-scheduler-api:5008/ \ | swh-scheduler -c remote -u http://swh-scheduler-api:5008/ \ | ||||
task add swh-lister-gitlab-full -p oneshot instance=https://0xacab.org | task add swh-lister-gitlab-full -p oneshot api_baseurl=https://0xacab.org/api/v4 | ||||
Created 1 tasks | Created 1 tasks | ||||
Task 12 | Task 12 | ||||
Next run: just now (2018-12-19 14:58:49+00:00) | Next run: just now (2018-12-19 14:58:49+00:00) | ||||
Interval: 90 days, 0:00:00 | Interval: 90 days, 0:00:00 | ||||
Type: swh-lister-gitlab-full | Type: swh-lister-gitlab-full | ||||
Policy: oneshot | Policy: oneshot | ||||
Args: | Args: | ||||
Keyword args: | Keyword args: | ||||
instance: https://0xacab.org | api_baseurl=https://0xacab.org/api/v4 | ||||
``` | ``` | ||||
This will insert a new task in the scheduler. To list existing tasks for a | This will insert a new task in the scheduler. To list existing tasks for a | ||||
given task type: | given task type: | ||||
``` | ``` | ||||
$ docker-compose run swh-scheduler-api \ | $ docker-compose run swh-scheduler-api \ | ||||
swh-scheduler -c remote -u http://swh-scheduler-api:5008/ \ | swh-scheduler -c remote -u http://swh-scheduler-api:5008/ \ | ||||
task list-pending swh-lister-gitlab-full | task list-pending swh-lister-gitlab-full | ||||
Found 1 swh-lister-gitlab-full tasks | Found 1 swh-lister-gitlab-full tasks | ||||
Task 12 | Task 12 | ||||
Next run: 2 minutes ago (2018-12-19 14:58:49+00:00) | Next run: 2 minutes ago (2018-12-19 14:58:49+00:00) | ||||
Interval: 90 days, 0:00:00 | Interval: 90 days, 0:00:00 | ||||
Type: swh-lister-gitlab-full | Type: swh-lister-gitlab-full | ||||
Policy: oneshot | Policy: oneshot | ||||
Args: | Args: | ||||
Keyword args: | Keyword args: | ||||
instance: https://0xacab.org | api_baseurl=https://0xacab.org/api/v4 | ||||
``` | ``` | ||||
To list all existing task types: | To list all existing task types: | ||||
``` | ``` | ||||
$ docker-compose run swh-scheduler-api \ | $ docker-compose run swh-scheduler-api \ | ||||
swh-scheduler -c remote -u http://swh-scheduler-api:5008/ \ | swh-scheduler -c remote -u http://swh-scheduler-api:5008/ \ | ||||
task --list-types | task --list-types | ||||
▲ Show 20 Lines • Show All 74 Lines • Show Last 20 Lines |
"one shot"