Page MenuHomeSoftware Heritage

Cleanup deprecated configuration code in swh modules
Open, HighPublic

Description

Once every swh modules have been deployed, we should cleanup the tasks submodule's collaborator (lister/loader/indexer)'s no longer useful configuration.

Once again, one step closer to T1410, T826.

Also cleanup the remaining dangling debian/ tree arborescence in every module's master branch (debian packaging in swh modules starts in debian/unstable-swh branch now).
Most if not all of the debian packaging should have been fixed once T1525 is done.

Then package and redeploy.

Related Objects

Event Timeline

ardumont triaged this task as High priority.Feb 16 2019, 11:37 AM
ardumont created this task.
ardumont updated the task description. (Show Details)Feb 16 2019, 11:40 AM
ardumont updated the task description. (Show Details)Feb 19 2019, 2:37 PM
ardumont removed ardumont as the assignee of this task.Jul 3 2019, 3:26 PM
ardumont added a comment.EditedMon, Sep 14, 5:27 PM

I recall doing all the extra work needed to move away from the SWHConfig in
production (and use SWH_CONFIG_FILENAME declared in the systemd service
definition T1531). The docker-compose also declares the SWH_CONFIG_FILENAME
variable as well (check environment > SWH_CONFIG_FILENAME entry). [1] [2]

So remaining deprecated code should be centralized around moving away from the
SWHConfig mixin inheritance. Here is what needs to be dropped/migrated then:

$SWH_ENVIRONMENT_HOME/swh-core/swh/core/config.py:class SWHConfig:
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/config.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/config.py:class APIConfig(SWHConfig):
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/loader/checker.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/loader/checker.py:class DepositChecker(SWHConfig):
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/client.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/client.py:class BaseApiDepositClient(SWHConfig):
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/indexer.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/indexer.py:class BaseIndexer(SWHConfig, metaclass=abc.ABCMeta):
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/rehash.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/rehash.py:class RecomputeChecksums(SWHConfig):
$SWH_ENVIRONMENT_HOME/swh-lister/swh/lister/core/lister_base.py:class ListerBase(abc.ABC, config.SWHConfig):
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/core/loader.py:class BaseLoader(config.SWHConfig, metaclass=ABCMeta):
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/loader.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/loader.py:        self.config = SWHConfig.parse_config_file()

[1] prod/staging: https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/?grep=SWH_CONFIG_FILENAME
[2] docker: https://forge.softwareheritage.org/source/swh-environment/browse/master/?grep=SWH_CONFIG_FILENAME