Page MenuHomeSoftware Heritage
Create Task
Maniphest T1181

pypi: Schedule ingestion
Closed, MigratedEdits Locked
Actions

Description

Once everything is developed, packaged, puppetized, trigger the ingestion:

  • scheduler updates (manifest, scheduler task type)
  • pypi lister (manifest, lister-pypi db)
  • pypi loader (manifest)
  • swh-workers: Deploy swh-loader-pypi service
  • saatchi: Deploy scheduler update
  • scheduler: Add task_type origin-update-pypi
  • Add scheduler task for listing pypi (and let the magic happen)

Related Objects
Search...

Event Timeline

ardumont triaged this task as Normal priority.Sep 6 2018, 5:31 PM
ardumont created this task.
ardumont renamed this task from pypi: Trigger listing task to pypi: Schedule ingestion.Sep 6 2018, 5:37 PM
ardumont updated the task description. (Show Details)
ardumont mentioned this in rSPSITE72d4a5354e7e: data/location: Deploy PyPI loader to swh-workers.Sep 19 2018, 7:41 PM
ardumont changed the task status from Open to Work in Progress.Sep 19 2018, 7:52 PM
ardumont updated the task description. (Show Details)
ardumont added a comment.Sep 20 2018, 9:47 AM

Schedule the lister-pypi:

swhscheduler@saatchi:~$ echo swh-lister-pypi | python3 -m swh.scheduler.cli task schedule -c type -
Created 1 tasks

Task 101693919
  Next run: just now (2018-09-20 07:46:32+00:00)
  Interval: 1 day, 0:00:00
  Type: swh-lister-pypi
  Policy: recurring
  Args:
  Keyword args:
ardumont updated the task description. (Show Details)EditedSep 20 2018, 9:47 AM

Status:

swhscheduler@saatchi:~$ python3 -m swh.scheduler.cli task list-pending -t swh-lister-pypi
Found 1 tasks

Task 101693919
  Next run: 2 minutes ago (2018-09-20 07:46:32+00:00)
  Interval: 1 day, 0:00:00
  Type: swh-lister-pypi
  Policy: recurring
  Args:
  Keyword args:
ardumont added a comment.Sep 20 2018, 9:52 AM

Now, it's scheduled. Just need to wait for the swh-scheduler-runner.service to finish its loop on task_types.

ardumont closed this task as Resolved.Sep 20 2018, 11:17 AM
ardumont claimed this task.
gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 9:58 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal