Page MenuHomeSoftware Heritage

Investigate scheduling policy for fsfe's gitea
Open, HighPublic

Description

Received an email from the fsfe wondering why their gitea seems to not have been visited since the first ingestion (29th of september).

Event Timeline

douardda created this task.

The listing task has been disabled, I think because of failures in the last executions:

swh scheduler task list -t list-gitea-incremental -i 337348713 -r
Found 1 tasks

Task 337348713
  Next run: févr. 08 (2021-02-08T22:02:24.327663+00:00)
  Interval: 1 day, 0:00:00
  Type: list-gitea-incremental
  Policy: recurring
  Status: disabled
  Priority: 
  Args:
  Keyword args:
    limit: 100
    url: 'https://git.fsfe.org/api/v1/'

  Executions:
    648ad06a-21b4-4e28-b219-976e99575c36 [failed]
      scheduled: 2021-02-07T22:02:23.998635+00:00 [None:2021-02-07T22:02:24.327663+00:00]
    2fc548a9-f983-44a8-8b9d-2d12752012db [failed]
      scheduled: 2021-02-06T22:02:19.506380+00:00 [None:2021-02-06T22:02:20.193159+00:00]
    1ded3d32-2bfa-4c7e-9292-c80caecd6907 [failed]
      scheduled: 2021-02-05T22:02:10.029705+00:00 [None:2021-02-05T22:02:10.241075+00:00]
    8690aa43-46d4-4a97-af6b-5450bcf78738 [failed]
      scheduled: 2021-02-04T22:02:06.864909+00:00 [None:2021-02-04T22:02:06.969813+00:00]
    865b67f1-95ba-43cc-9c58-d3af0a114ef5 [uneventful]
      scheduled: 2021-02-03T22:02:02.418506+00:00 [None:2021-02-03T22:02:05.791897+00:00]
    338125ce-6c0b-4af7-87e4-97d7c64d7366 [uneventful]
      scheduled: 2021-02-02T22:01:50.436796+00:00 [2021-02-02T22:01:50.757913+00:00:2021-02-02T22:01:55.200607+00:00]
    bb46d1ad-ef94-4371-a888-bf9def05bbfc [uneventful]
      scheduled: 2021-02-01T22:01:44.966546+00:00 [2021-02-01T22:01:45.529451+00:00:2021-02-01T22:01:49.599089+00:00]
[...]

The loading tasks created during this first listing were oneshot tasks. So they have been modified to recurring tasks with something like:

update task set policy='recurring', status='next_run_not_scheduled', priority=null  where type = 'load-git' and arguments->'kwargs'->>'url' like 'https://git.fsfe.org/%'  and policy='oneshot' and id >= 337024433 and id <= 337348712 ;

Note: I've limited the range of ids to the ones I previously queried to make the update faster.

Note sure when they are going to be scheduled however...