Differential D6557 Diff 23831 sysadm/puppet/howto-manage-third-party-modules.rst

Changeset View

Standalone View

sysadm/puppet/howto-manage-third-party-modules.rst

	.. _puppet_third_party:			.. _puppet_integration_of_third_party_puppet_modules:

	How to manage Third-Party modules			How to manage Third-Party modules
	=================================			=================================

	.. todo::			Integration of third party puppet modules
	This page is a work in progress. Please refer to the `existing documentation			-----------------------------------------
	<https://forge.softwareheritage.org/source/puppet-environment/>`_ and `the
	wiki page <https://wiki.softwareheritage.org/wiki/Puppet_setup>`_.			We mirror external repositories to our own forge, to avoid having external dependencies
				in our deployment.

				In the swh-site ``Puppetfile``, we pin the installation of those modules to the
				highest version (that works with our current puppet/facter version), by using the :ref
				specifier.

				.. _adding_a_new_external_puppet_module:

				Adding a new external puppet module
				~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

				In the puppet-environment repository, the ``bin/import-puppet-module`` takes care of
				the following tasks:

				- Getting metadata from the `Puppet forge <https://forge.puppetlabs.com/>`_ for the
				module (description, upstream git URL)
				- Cloning the repository
				- Creating a mirror repository on the Software Heritage forge, with the proper
				permissions and metadata (notably the Sync to GitHub flag)
				- Pushing the clone to the forge
				- Updating the .mrconfig and .gitignore files

				To be able to use the script, you need:

				- Be a member of the `System Administrators
				<https://forge.softwareheritage.org/project/members/7/>`_ Phabricator group
				- Have the :ref:`Arcanist <swh-devel:arcanist-configuration>` API key setup
				- A pair of python dependencies: ``python3-phabricator`` and ``python3-requests`` (pull
				them from testing if needed).

				Example usage to pull the `elastic/elasticsearch
				<https://forge.puppetlabs.com/elastic/elasticsearch>`_ module

				::

				bin/import-module elastic-elasticsearch
				git diff # review changes
				git add .mrconfig .gitignore
				git commit -m "Add the elastic/elasticsearch module"
				git push

				Once the module is added, you need to register it in the swh-site `Puppetfile
				<https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/Puppetfile>`_.

				You should also check in the module metadata whether any dependencies need importing as
				well, which you should do using the same procedure.

				.. _updating_external_puppet_modules:

				Updating external puppet modules
				~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

				There's two sides of this coin:

				.. _updating_our_git_clone_of_external_puppet_modules:

				Updating our git clone of external puppet modules
				^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

				The puppet-environment ``.mrconfig`` file has a ``pullup`` command which does the
				right thing.

				To update all clones:

				::

				mr -j4 pullup

				.. _upgrading_external_puppet_modules:

				Upgrading external puppet modules
				^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

				Upgrading external puppet modules happens manually.

				In the puppet-environment repository, the ``bin/check-module-updates`` script compares
				the Puppetfile and the local clones and lists the available updates. (depends on ``ruby
				r10k``).

				On a staging branch of the swh-site repository, update the ``:ref`` value for the
				module in the ``Puppetfile`` to the latest tag. You can then run ``octocatalog-diff`` on
				a few relevant servers and look for changes.