Page MenuHomeSoftware Heritage

lister-sourceforge: Activate sourceforge origins when listed
Closed, ResolvedPublic

Description

sourceforge origins are scheduled specifically due to upstream's wishes at the time. Now
that the gist of the ingestion is done for those origins, we should get back to a
standard scheduling.

That means:

  • reverting the commit that deactivates those origins in swh.lister
  • stop the runners that specifically schedule those deactivated origins [1]
  • sql: migrate back from enabled=false to enabled=true in the scheduler db (table listed_origins)
  • let "standard" next-gen runner pick those up and schedule those origins

[1] D5818 <- runners in worker17 running that diff in venv

Event Timeline

ardumont triaged this task as High priority.Aug 6 2021, 4:09 PM
ardumont created this task.
ardumont changed the task status from Open to Work in Progress.Oct 14 2021, 3:00 PM
ardumont moved this task from Weekly backlog to in-progress on the System administration board.

Lister got adapted to actually activate the sourceforge origins.
That lister got deployed.

A full listing on sourceforge is actually ongoing.
This should activate back the sourceforge origins.
Which should in turn be scheduled normally for ingestion.

Sourceforge origins are progressively activated back and getting ingested along the way [1].

[1] https://grafana.softwareheritage.org/goto/ZF0tuGOnk?orgId=1