swh-mirror tests are quite involved as they deploy a replicated Software
Heritage Archive in the form of a Docker swarm.
This new `swh-mirror/run-tests` job assumes that Jenkins running has
access to a Docker engine. It needs:
1. to be able to build images;
2. to be configured as part of a swarm;
3. to have at least one node with the following labels:
org.softwareheritage.mirror.volumes.storage-db=true
org.softwareheritage.mirror.volumes.web-db=true
org.softwareheritage.mirror.volumes.objstorage=true
org.softwareheritage.mirror.volumes.redis=true
The pipeline will first create up-to-date Docker images used by the
swarm (using Jenkins host) and then run `pytest` in a Docker container
using `swh-jenkins/tox`.
As it depends on network resources, the test can drag for quite long.
The timeout is currently set at one hour. In most cases the pipeline
will be done in 15-20 minutes.
Care is taken both in the test code and within the Jenkins pipeline to
remove resources related to the Docker swarm (stack, containers, configs
and volumes) on teardown.
FIXME: I was unable to use an include with 'includes/agent-docker.groovy.j2'
to define the running agent for the Python tests. jenkins-job-builder
failed with “TypeError: expected string or bytes-like object”.
It looks related to variables used by the included template but
it was unclear why it could not find them. The template is inlined as a
workaround.
The job will need the changes proposed as D8634 to work.