diff --git a/README.md b/README.md index fde560e..dc3ee98 100644 --- a/README.md +++ b/README.md @@ -1,65 +1,170 @@ Software Heritage Puppet environment ==================================== This repository contains the metadata for Software Heritage's puppet infrastructure git repositories. The repositories are managed using [myrepos][1] (see the .mrconfig file), and the `mr` command. [1]: http://myrepos.branchable.com/ As our .mrconfig file contains "untrusted" checkout commands (to setup the upstream repositories of our mirrors of third-party modules), you need to add the .mrconfig file to your ~/.mrtrust file: readlink -f .mrconfig >> ~/.mrtrust You can then checkout the repositories using `mr up`. For periodic updates after initial setup, you can use the `bin/update` helper: cd puppet-environment bin/update Module Layout ------------- We use dynamic environments, and the role/profiles puppet workflow as demonstrated by the following series of articles: - http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-1/ - http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-2/ - http://garylarizza.com/blog/2014/02/18/puppet-workflow-part-3/ - http://garylarizza.com/blog/2014/03/07/puppet-workflow-part-3b/ - http://garylarizza.com/blog/2014/10/24/puppet-workflows-4-using-hiera-in-anger/ Our main manifests are present in the `swh-site` repository. Each branch of that repository corresponds to an environment in the puppet workflow presented. This repository contains the Puppetfile referencing all the modules, the main manifest file `manifests/site.pp`, and the hiera `data` directory, as well as the two "site-modules" for `profile`s and `role`s. Our setup mirrors the git repositories of third-party Puppet modules on the Software Heritage git server --- this is to avoid reliance on 3rd party *hosting* services in order to be able to deploy. We add an upstream remote to the repositories through our mr configuration. Deployment ---------- Deployment happens on the `pergamon.softwareheritage.org` server, using our custom deployment script: sudo /etc/puppet/environments/production/deploy.sh This updates the dynamic environments according to the contents of the branches of the git repository, and the Puppetfile inside. For each third-party module, we pin the module definition in the Puppetfile to a specific tag or revision. Our specific deploy script also fetches private repositories and merges them with the public r10k setup. + +Local puppet manifest diffing with octocatalog-diff +--------------------------------------------------- + +puppet-environment contains the whole scaffolding to be able to use [octocatalog-diff][2] on our manifests. This allows for quick(er) local iterations while developing complex puppet manifests. +Dependencies + +[2]: https://github.com/github/octocatalog-diff + +You need the following packages installed on your machine: + + r10k octocatalog-diff puppet + +### Running + +The `bin/octocatalog-diff` script allows diffing the manifests between two environments (that is, between two branches of the *swh-site* repository. By default it diffs between production and staging. + +Default usage: + + bin/octocatalog-diff pergamon + +Limitations + +Our setup for octocatalog-diff doesn't support exported resources, so you won't see your fancy icinga checks there. + +Integration of third party puppet modules +----------------------------------------- + +We mirror external repositories to our own forge, to avoid having external dependencies in our deployment. + +In the `swh-site/Puppetfile`, we pin the installation of those modules to the highest version (that works with our current puppet/facter version), by using the `:ref` specifier. + +### Adding a new external puppet module + + +In the *puppet-environment* repository, the `bin/import-puppet-module` takes care of the following tasks: + +* Getting metadata from the Puppet forge for the module (description, upstream git URL) +* Cloning the repository +* Creating a mirror repository on the Software Heritage forge, with the proper permissions and metadata (notably the Sync to GitHub flag) +* Pushing the clone to the forge +* Updating the `.mrconfig` and `.gitignore` files to know the new repository + +To be able to use the script, you need to : + +* Be a member of the System Administrators Phabricator group +* Have the Arcanist API key setup +* A pair of python dependencies : `python3-phabricator` and `python3-requests` (pull them from testing if needed). + +Example usage to pull the `elastic/elasticsearch` module + + bin/import-module elastic-elasticsearch + git diff # review changes + git add .mrconfig .gitignore + git commit -m "Add the elastic/elasticsearch module" + git push + +Once the module is added, you need to register it in `swh-site/Puppetfile`. + +You should also check in the module metadata whether any dependencies need importing as well, which you should do using the same procedure. + +### Updating external puppet modules + +There's two sides of this coin: + +#### Updating our git clone of external puppet modules + +The *puppet-environment* `.mrconfig` file has a pullup command which does the right thing. + +To update all clones: + + mr -j4 pullup + +#### Upgrading external puppet modules + +Upgrading external puppet modules happens manually. + +In the *puppet-environment* repository, the `bin/check-module-updates` script compares the Puppetfile and the local clones and lists the available updates. (depends on `ruby r10k`). + +On a staging branch of the *swh-site* repository, update the `:ref` value for the module in the `Puppetfile` to the latest tag. You can then run `octocatalog-diff` on a few relevant servers and look for changes. + +Deploy workflow +---------------- + +### Semi-automated + + you@localhost$ # hack on puppet Git repo + you@localhost$ rake validate + you@localhost$ git commit + you@localhost$ git push + you@localhost$ cd puppet-environment + you@localhost$ bin/deploy-on machine1 machine2... + +Remember to pass `--apt` to `bin/deploy-on` if freshly uploaded Software Heritage packages are to be deployed. Also, `bin/deploy-on --help` is your friend. + +### Manual + + you@localhost$ # hack on puppet Git repo + you@localhost$ rake validate + you@localhost$ git commit + you@localhost$ git push + you@pergamon$ sudo swh-puppet-master-deploy + you@machine$ sudo apt-get update # if a new or updated version of a Debian package needs deploying + you@machine$ sudo swh-puppet-test # to test/review changes + you@machine$ sudo swh-puppet-apply # to apply