Page MenuHomeSoftware Heritage

staging: Migrate remaining static services to elastic workers
Closed, MigratedEdits Locked


  1. Creating venvs/docker images for remaining services:
  2. D8500: swh.loader.mercurial
  3. swh.loader.cvs
  4. swh.loader.svn
  5. swh.vault.cookers
  6. swh.deposit.checkers
  7. swh.indexer journal client
  1. Reference service in staging charts:
  2. D8501: mercurial
  3. loader-cvs
  4. loader-svn
  5. loader-archive
  6. loader-cran
  7. loader-debian
  8. loader-deposit
  9. loader-nixguix
  10. loader-opam
  11. remaining (staging) listers
  12. T4540#91856: vault cooker
  13. T4540#92071: deposit checker
  14. swh indexer journal client
  1. Decomission loader in staging static workers:
  2. loader-archive
  3. loader-cran
  4. loader-cvs
  5. loader-debian
  6. loader-deposit
  7. loader-svn
  8. loader-git
  9. loader-bzr
  10. loader-highpriority
  11. loader-maven
  12. loader-mercurial
  13. loader-npm
  14. loader-nixguix
  15. loader-opam
  16. loader-pypi
  17. remaining staging listers
  18. vault cookers
  19. deposit checker
  20. low hanging fruit indexers: swh indexer journal client
  21. migrate remaining fossology license indexer

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ardumont renamed this task from staging: Migrate remaining static loader services to elastic workers to staging: Migrate remaining static loader/lister services to elastic workers.Sep 22 2022, 9:45 AM

Remains a bunch of services not existing yet in the elastic stack (vault cooker, deposit checkers and indexer journal clients).
So the plan would be to integrate those in the elastic stack.

And then we could decomission all staging static workers altogether.

ardumont renamed this task from staging: Migrate remaining static loader/lister services to elastic workers to staging: Migrate remaining static services to elastic workers.Sep 26 2022, 2:46 PM

Cookers migrated:

cookers [2022-09-26 14:48:17,948: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHCookingTask[722de41d-0f88-45f7-ad06-3fb65f3978e1] received
cookers [2022-09-26 14:48:54,924: INFO/ForkPoolWorker-1] Task swh.vault.cooking_tasks.SWHCookingTask[722de41d-0f88-45f7-ad06-3fb65f3978e1] succeeded in 36.83566682902165s: None
cookers [2022-09-26 14:54:28,340: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHCookingTask[0926cf58-6001-496c-99aa-d74f5c57175d] received
cookers [2022-09-26 14:54:30,130: INFO/ForkPoolWorker-2] Task swh.vault.cooking_tasks.SWHCookingTask[0926cf58-6001-496c-99aa-d74f5c57175d] succeeded in 1.7597111140494235s: None
cookers [2022-09-26 14:55:33,509: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHCookingTask[839c03c3-159f-46e9-9f19-56817866321a] received
cookers [2022-09-26 14:55:33,516: INFO/MainProcess] Task swh.vault.cooking_tasks.SWHCookingTask[c6ad1f3c-31aa-4ef7-9f90-093d56011ddf] received
cookers [2022-09-26 14:56:34,268: INFO/ForkPoolWorker-3] Task swh.vault.cooking_tasks.SWHCookingTask[839c03c3-159f-46e9-9f19-56817866321a] succeeded in 60.72709982498782s: None
cookers [2022-09-26 14:56:41,961: INFO/ForkPoolWorker-4] Task swh.vault.cooking_tasks.SWHCookingTask[c6ad1f3c-31aa-4ef7-9f90-093d56011ddf] succeeded in 3.1700519190053456s: None

Deposit checkers migrated. Trigger icinga alert check on deposit, it went through the elastic workers [1] so done!


│ checker-deposit [2022-09-27 10:25:57,868: INFO/MainProcess] checker@checker-deposit-95b97647f-455jc ready.                                                                                                                                  │
│ checker-deposit [2022-09-27 10:26:40,799: INFO/MainProcess] Task swh.deposit.loader.tasks.ChecksDepositTsk[8dcc56b9-2ee8-48bc-a75f-126c83fdb871] received                                                                                   │
│ checker-deposit [2022-09-27 10:26:42,198: INFO/ForkPoolWorker-1] Task swh.deposit.loader.tasks.ChecksDepositTsk[8dcc56b9-2ee8-48bc-a75f-126c83fdb871] succeeded in 1.3463120240485296s: {'status': 'eventful'}                              │
│ checker-deposit [2022-09-27 10:27:46,933: INFO/MainProcess] Task swh.deposit.loader.tasks.ChecksDepositTsk[a6840e62-ae1d-4529-810d-c372f68c0fc4] received                                                                                   │
│ checker-deposit [2022-09-27 10:27:48,038: INFO/ForkPoolWorker-2] Task swh.deposit.loader.tasks.ChecksDepositTsk[a6840e62-ae1d-4529-810d-c372f68c0fc4] succeeded in 1.0527836499968544s: {'status': 'eventful'}
ardumont updated the task description. (Show Details)
ardumont updated the task description. (Show Details)
ardumont moved this task from deployed/landed/monitoring to done on the System administration board.