Page MenuHomeSoftware Heritage

templates/cypress: Run tests in parallel using sorry-cypress
ClosedPublic

Authored by anlambert on Aug 26 2022, 5:28 PM.

Details

Summary

In order to gain some time when executing Jenkins jobs for swh-web,
execute cypress tests in parallel using 4 runners thanks to the
sorry-cypress tool.

To do so, we use the docker features of Jenkins pipeline.
One sidecar container is created to execute the sorry-cypress server
and parallel tests are executed in another container spawning 4
cypress processes. The communication between the two containers
is done through a dedicated docker network.

Regarding code coverage, we now need to merge the outputs produced
by the 4 runners before generating reports.

This change brings a 2x speedup on overall swh-web build jobs execution
(when a single job is executed, speedup is lesser when multiple jobs
are executed in parallel but still significative).

Also restrict the maximum number of concurrent builds per job to 3
in order to avoid executing too many cypress processes at the same
time as they consume quite a lot of resources on the Jenkins node.

Related to T3423

Depends on D8368

Test Plan

I have created a new pipeline for swh-web on our Jenkins instance
with these changes. This seems to work great, even when multiple jobs are executed in parallel.

I will continue to battle test this next week.

Diff Detail

Repository
rCJSWH Jenkins jobs
Branch
cypress-parallel-tests
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 31241
Build 48868: arc lint + arc unit

Event Timeline

Based on my tests on our Jenkins instance, swh-web build jobs with the changes in that diff become unstable when there is more
than three jobs executed in parallel (as there is a lot of cypress processes consuming a lot of resources on the Jenkins node).

So I think we should restrict the number of concurrent builds for the swh-web pipeline to two or three using the Throttle Concurrent Builds
Jenkins plugin.

Update:

  • Rebase
  • Improve cypress step implementation
  • Limit max number of concurrent builds per job to 3
olasd added a subscriber: olasd.

Cool stuff, thanks!

This revision is now accepted and ready to land.Sep 5 2022, 2:39 PM