Page MenuHomeSoftware Heritage

Big pile of fixes to adapt to current HEAD of core,scheduler,storage etc.
ClosedPublic

Authored by douardda on Thu, Jan 31, 5:29 PM.

Details

Summary
  • Indexers have been refactored so we now only one celery worker capable of doing all the indexers' work.
  • Fix the celery worker's hostname
  • Update config of the scheduler adapt to latest swh-scheduler's refactorings.
  • Add a prefix in celery workers' hostnames to prevent confusion eg. with pypi for which both the lister and the loader are named just 'pypi'.
  • Ensure pip, setuptools and wheel are up to date at image build time
  • Fix a typo in indexer's Dockerfile
  • Update config system for scheduler services
  • Workers' celery config is now managed as the [celery] section of the main config file

These 2 items below do not exactly fit in this pile of fixes, to be honnest...
Blame lazy me!

  • Reduce the slicing for objstorage it's probably best in the context of adocker test env...
  • Small fix for the prometheus service

    looks something has changed recently in the prom docker, not sure how to know what exactly, it's unclear to ne how the prom/prometheus published on docker's hub is actually built.

Diff Detail

Repository
rCDFD Dockerfiles for developers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

douardda created this revision.Thu, Jan 31, 5:29 PM
vlorentz accepted this revision.Thu, Jan 31, 5:55 PM
This revision is now accepted and ready to land.Thu, Jan 31, 5:55 PM
ardumont accepted this revision.Fri, Feb 1, 11:29 AM
This revision was automatically updated to reflect the committed changes.