.ini config
ClosedPublic
Actions

Authored by ardumont on Feb 19 2019, 7:16 PM.

Details

Reviewers

olasd
vlorentz
douardda

Group Reviewers

Reviewers

Commits

rSPSITE87892018153f: worker/*: Remove duplicated and no longer used setup
rSPSITEb509a9af35fb: instance: Remove no longer used parameters
rSPSITE47daf640ffff: deploy/worker: Remove no longer used workers
rSPSITEc8416ab51dda: worker: Remove no longer used worker/<service>.ini config

Summary

First step towards cleaning up unused stuffs.
Then deploy and make sure everything is still fine.
Then refactor some more instance.pp (generic systemd unit for our celery workers).

deploy/worker: Remove no longer used workers
worker/*: Remove duplicated and no longer used setup
instance: Remove no longer used parameters

Related T1531

Test Plan

bin/octocatalog-diff:

Found host worker01.softwareheritage.org
*** Running octocatalog-diff on host worker01.softwareheritage.org
diff origin/production/worker01.softwareheritage.org current/worker01.softwareheritage.org
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/lister_debian.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/lister_github.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/lister_gitlab.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/lister_pypi.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/loader_debian.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/loader_deposit.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/loader_git.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/loader_mercurial.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/loader_pypi.ini][0m
*******************************************
[0;31;49m- File[/etc/softwareheritage/worker/loader_svn.ini][0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[lister_debian] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.lister.debian.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_lister_debian"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[lister_github] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.lister.github.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_lister_github_discover", "swh_lister_github_refresh"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[lister_gitlab] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.lister.gitlab.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_lister_gitlab_discover", "swh_lister_gitlab_refresh"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[lister_pypi] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.lister.pypi.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_lister_pypi_refresh"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[loader_debian] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.loader.debian.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_loader_debian"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[loader_deposit] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.deposit.loader.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_checker_deposit", "swh_loader_deposit"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[loader_git] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.loader.git.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_loader_git"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[loader_mercurial] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.loader.mercurial.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_loader_mercurial", "swh_loader_mercurial_archive"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[loader_pypi] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.loader.pypi.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_loader_pypi"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
  Profile::Swh::Deploy::Worker::Instance[loader_svn] =>
   parameters =>
     task_broker =>
[0;31;49m      - amqp://swhconsumer:swh::deploy::worker::task_broker::password@rabbitmq:5672//[0m
     task_modules =>
[0;31;49m      - ["swh.loader.svn.tasks"][0m
     task_queues =>
[0;31;49m      - ["swh_loader_svn", "swh_loader_svn_mount_and_load"][0m
     task_soft_time_limit =>
[0;31;49m      - 0[0m
*******************************************
*** End octocatalog-diff on worker01.softwareheritage.org

Diff Detail

Repository

rSPSITE puppet-swh-site

Branch

remove-no-longer-used-setup

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 4323
Build 5710: arc lint + arc unit

Event Timeline

ardumont created this revision.Feb 19 2019, 7:16 PM

Harbormaster completed remote builds in B4310: Diff 3633.Feb 19 2019, 7:16 PM

ardumont marked an inline comment as done.Feb 19 2019, 7:17 PM

ardumont added inline comments.

data/defaults.yaml
1555	celery setup is inlined in the loader config now.

ardumont edited the summary of this revision. (Show Details)Feb 19 2019, 7:35 PM

ardumont edited the test plan for this revision. (Show Details)

ardumont mentioned this in D1154: Refactor: synchronize workers' service names with configuration yaml key.Feb 19 2019, 8:31 PM

ardumont added a project: Puppet recipes.

ardumont mentioned this in D1155: worker/instance: Refactor worker systemd initialization.Feb 19 2019, 9:14 PM

Looks fine to me considering ::profile::swh::deploy::worker::base is properly setup to clean up the /etc/softwareheritage/worker directory (as by itself, removing a resource definition does not remove the resource).

I'm guessing that directory could just be removed at this point.

This revision is now accepted and ready to land.Feb 20 2019, 8:45 AM

ardumont edited the summary of this revision. (Show Details)Feb 20 2019, 10:27 AM

ardumont mentioned this in D1156: Remove implicit configuration from the metadata indexers..Feb 20 2019, 11:06 AM

Prepare merge

Harbormaster completed remote builds in B4322: Diff 3645.Feb 20 2019, 11:08 AM

Fix branch name (T.T)

Harbormaster completed remote builds in B4323: Diff 3646.Feb 20 2019, 11:09 AM

Landed (tip: 54036f6).

ardumont mentioned this in D1158: manifest: Latest refactoring/cleanup .Feb 20 2019, 12:12 PM

ardumont mentioned this in rSPSITEdcc4b60c2a7d: deploy/worker/base: Remove no longer used config directory.Feb 20 2019, 6:08 PM