Page MenuHomeSoftware Heritage

staging: Deploy scheduler v0.17
Closed, ResolvedPublic

Description

Plan:

  • Package swh.scheduler v0.17.0
  • Upgrade python3-swh.scheduler on scheduler0.staging
  • Migrate db version from 28 to 30 (impacted tables: origin_visit_stats, scheduler_metrics)
  • Restart scheduler services [1]
  • Ensure the journal client is doing its new job

[1]

systemctl restart gunicorn-swh-scheduler \
  swh-scheduler-listener.service \
  swh-scheduler-runner.service \
  swh-scheduler-runner-priority.service \
  swh-scheduler-journal-client.service

Event Timeline

ardumont created this task.
ardumont changed the task status from Open to Work in Progress.Aug 6 2021, 12:11 PM
ardumont moved this task from Backlog to in-progress on the System administration board.
ardumont updated the task description. (Show Details)

Ensure the journal client is doing its new job

It does, next_position_offset, last_visit, last_successful, ... filled in along the ingestions.

15:09:54 swh-scheduler@db1:5432=> select now(), * from origin_visit_stats where last_visit is not null order by last_visit desc limit 1;
+-[ RECORD 1 ]--------------+--------------------------------------------------+
| now                       | 2021-08-06 13:09:23.117781+00                    |
| url                       | https://pypi.org/project/collective.siteimprove/ |
| visit_type                | pypi                                             |
| last_snapshot             | \xd472985b96eb631792044170953eb1863b180147       |
| last_scheduled            | (null)                                           |
| next_visit_queue_position | 2021-08-10 16:20:00.909152+00                    |
| next_position_offset      | 5                                                |
| successive_visits         | 1                                                |
| last_successful           | 2021-08-06 13:09:22.423643+00                    |
| last_visit                | 2021-08-06 13:09:22.423643+00                    |
| last_visit_status         | successful                                       |
+---------------------------+--------------------------------------------------+

Time: 548.660 ms
15:09:57 swh-scheduler@db1:5432=> select now(), * from origin_visit_stats where last_visit is not null order by last_visit desc limit 1;
+-[ RECORD 1 ]--------------+--------------------------------------------+
| now                       | 2021-08-06 13:09:24.757893+00              |
| url                       | https://pypi.org/project/heihama/          |
| visit_type                | pypi                                       |
| last_snapshot             | \x1d9ba6535abd984635982e3b040ed4a05f0c1742 |
| last_scheduled            | (null)                                     |
| next_visit_queue_position | 2021-08-10 09:01:10.935763+00              |
| next_position_offset      | 5                                          |
| successive_visits         | 1                                          |
| last_successful           | 2021-08-06 13:09:23.318129+00              |
| last_visit                | 2021-08-06 13:09:23.318129+00              |
| last_visit_status         | successful                                 |
+---------------------------+--------------------------------------------+

Time: 532.187 ms