Page MenuHomeSoftware Heritage

Big pile of fixes to adapt to current HEAD of core,scheduler,storage etc.
ClosedPublic

Authored by douardda on Jan 31 2019, 5:29 PM.

Details

Summary
  • Indexers have been refactored so we now only one celery worker capable of doing all the indexers' work.
  • Fix the celery worker's hostname
  • Update config of the scheduler adapt to latest swh-scheduler's refactorings.
  • Add a prefix in celery workers' hostnames to prevent confusion eg. with pypi for which both the lister and the loader are named just 'pypi'.
  • Ensure pip, setuptools and wheel are up to date at image build time
  • Fix a typo in indexer's Dockerfile
  • Update config system for scheduler services
  • Workers' celery config is now managed as the [celery] section of the main config file

These 2 items below do not exactly fit in this pile of fixes, to be honnest...
Blame lazy me!

  • Reduce the slicing for objstorage it's probably best in the context of adocker test env...
  • Small fix for the prometheus service

    looks something has changed recently in the prom docker, not sure how to know what exactly, it's unclear to ne how the prom/prometheus published on docker's hub is actually built.

Diff Detail

Repository
rCDFD Dockerfiles for developers
Branch
master
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 3917
Build 5129: arc lint + arc unit

Event Timeline

douardda created this revision.Jan 31 2019, 5:29 PM
vlorentz accepted this revision.Jan 31 2019, 5:55 PM
This revision is now accepted and ready to land.Jan 31 2019, 5:55 PM
ardumont accepted this revision.Feb 1 2019, 11:29 AM
This revision was automatically updated to reflect the committed changes.