Page MenuHomeSoftware Heritage
Create Task
Maniphest T2967

Write journal client subcribed to origin_visit_status topics
Closed, ResolvedPublic
Actions

Description

In order to fill in the new scheduler "origin_visit_stats" table.

plan:

  • cli: open new cli which reads configuration, instantiate journal client
  • which reads origin_visit_status topics
  • to fill-in the new origin_visit_stats table
  • deploy journal-client on staging
  • deploy journal-client on production

Revisions and Commits

rDSCH Scheduling utilities
AbandonedD4859 visit_stats: Update references when per event date is more recent
AbandonedD4853 visit_stats: Keep a reference to the last snapshot
AbandonedD4852 Start journal client to populate the origin_visit_stats model
D4890rDSCH9fb0dd6c7cc3 journal_client: Read visit_stats entries by batch out of the loop
D4888rDSCHd464b4cc1f9a scheduler: Make origin_visit_stats_get read multiple entries
D4876rDSCH9395aa0763b7 scheduler.cli.journal: Add `swh scheduler journal-client` cli
D4873rDSCH58ca79622db0 journal_client: Improve stats detection
D4866rDSCH608aa2075619 Populate origin_visit_stats table out of the origin_visit_status topic

Related Objects
Search...

Event Timeline

ardumont triaged this task as High priority.Jan 13 2021, 9:51 AM
ardumont created this task.
ardumont moved this task from Backlog to todo on the Sprint 2021 01 board.
ardumont changed the task status from Open to Work in Progress.Jan 13 2021, 11:38 AM
ardumont claimed this task.
ardumont moved this task from todo to in-progress on the Sprint 2021 01 board.
ardumont updated the task description. (Show Details)Jan 13 2021, 11:45 AM
ardumont added a revision: D4852: Start journal client to populate the origin_visit_stats model.Jan 13 2021, 1:08 PM
ardumont added a revision: D4853: visit_stats: Keep a reference to the last snapshot.Jan 13 2021, 3:37 PM
ardumont added a revision: D4859: visit_stats: Update references when per event date is more recent.Jan 13 2021, 6:44 PM
ardumont added a revision: D4866: Populate origin_visit_stats table out of the origin_visit_status topic.Jan 14 2021, 5:46 PM
ardumont added a commit: rDSCH608aa2075619: Populate origin_visit_stats table out of the origin_visit_status topic.Jan 15 2021, 2:45 PM
ardumont mentioned this in rDSCHa5fb291703d2: backend: Make origin_visit_stats_upsert a batch api.Jan 15 2021, 3:48 PM
ardumont added a revision: D4873: journal_client: Improve stats detection.Jan 15 2021, 4:57 PM
ardumont updated the task description. (Show Details)Jan 15 2021, 5:30 PM
ardumont added a revision: D4876: scheduler.cli.journal: Add `swh scheduler journal visit-stats` cli.Jan 18 2021, 3:36 PM
ardumont added a commit: rDSCH58ca79622db0: journal_client: Improve stats detection.Jan 19 2021, 10:22 AM
ardumont added a commit: rDSCH9395aa0763b7: scheduler.cli.journal: Add `swh scheduler journal-client` cli.Jan 19 2021, 12:09 PM
vsellier updated the task description. (Show Details)Jan 19 2021, 1:45 PM
vsellier changed the status of subtask T2978: Deploy visit-stats journal client on staging from Open to Work in Progress.Jan 19 2021, 1:48 PM
ardumont added a revision: D4888: scheduler: Make origin_visit_stats_get read multiple entries.Jan 19 2021, 6:32 PM
ardumont added a revision: D4890: journal_client: Read visit_stats entries by batch out of the loop.Jan 19 2021, 6:52 PM
ardumont added a commit: rDSCHd464b4cc1f9a: scheduler: Make origin_visit_stats_get read multiple entries.Jan 21 2021, 11:02 AM
ardumont added a commit: rDSCH9fb0dd6c7cc3: journal_client: Read visit_stats entries by batch out of the loop.
ardumont closed this task as Resolved.Jan 21 2021, 11:42 AM
ardumont moved this task from in-progress to code review on the Sprint 2021 01 board.
ardumont moved this task from code review to done on the Sprint 2021 01 board.
vsellier closed subtask T2978: Deploy visit-stats journal client on staging as Resolved.Jan 22 2021, 11:12 AM
vsellier updated the task description. (Show Details)Jan 25 2021, 10:56 AM
About · Developer information · Legal