Add a job running swh-mirror tests
swh-mirror tests are quite involved as they deploy a replicated Software
Heritage Archive in the form of a Docker swarm.
This new swh-mirror/run-tests job assumes that Jenkins running has
access to a Docker engine. It needs:
- to be able to build images;
- to be configured as part of a swarm;
- to have at least one node with the following labels:
org.softwareheritage.mirror.volumes.storage-db=true org.softwareheritage.mirror.volumes.web-db=true org.softwareheritage.mirror.volumes.objstorage=true org.softwareheritage.mirror.volumes.redis=true
The pipeline will first create up-to-date Docker images used by the
swarm (using Jenkins host) and then run pytest in a Docker container
using swh-jenkins/tox.
As it depends on network resources, the test can drag for quite long.
The timeout is currently set at one hour. In most cases the pipeline
will be done in 15-20 minutes.
Care is taken both in the test code and within the Jenkins pipeline to
remove resources related to the Docker swarm (stack, containers, configs
and volumes) on teardown.
The job will need the changes proposed as D8634 to work.