Execute cypress tests in parallel on Jenkins
Closed, MigratedEdits Locked
Actions

Assigned To

Authored By

	anlambert
	Jul 6 2021, 2:03 PM

Description

Cypress overall tests execution is quite long as they are sequentially ran by default.

In order to speedup tests execution, they can be run in parallel using multiple test executors.

Nevertheless, this requires to use the Cypress dashboard that cannot be self hosted and is quite limited for its free pricing plan.

Fortunately, it exists the sorry-cypress open source project enabling to run parallel cypress tests while being self hostable.
We should consider using it to speedup cypress tests on Jenkins.

Revisions and Commits

rDWAPPS Web applications
	D8387	rDWAPPSe96bc074f808 cypress: Use more reliable way to get global javascript variable
	D8367	rDWAPPSde37826d1446 cypress/origin-save: Add missing wait instructions in a test
	D8367	rDWAPPS506470cce04b cypress/support: Add missing wait instruction in global before hook
	D8326	rDWAPPS61b575db215d sqlite: Use multiple database files when running parallel cypress tests
	D8326	rDWAPPScb03344f8a58 package.json: Add cy2 development dependency
	D8326	rDWAPPS3adcbde7f785 cypress/admin: Add missing intercept in test implementation
	D8326	rDWAPPS4a8de541123b cypress.config.js: Add projectId
	D8326	rDWAPPSb7be6a88dce7 cypress/origin-save: Add wait instructions to fix flaky tests
	D8326	rDWAPPS4b83f99981a4 cypress/add-forge-now-request-create: Add missing wait instructions
rCJSWH Jenkins jobs
	D8328	rCJSWHddbafe952e63 templates/cypress: Run tests in parallel using sorry-cypress
	D8368	rCJSWHb00f888ceffb jobs/swh-pipeline: Enable to limit number of concurrent builds per job

Event Timeline

anlambert triaged this task as Normal priority.Jul 6 2021, 2:03 PM

anlambert created this task.

anlambert updated the task description. (Show Details)Jul 6 2021, 2:09 PM

After some hacks on the current Jenkins pipeline for swh-web to test if it was feasible, I managed to setup the use of sorry-cypress but could not run multiple cypress in parallel as Jenkins pipelines forbid nested parallel stages.
So this would require setting up a dedicated pipeline build job for cypress tests as currently Python and cypress tests are executed in parallel in a single job.

In T3423#67264, @anlambert wrote:

After some hacks on the current Jenkins pipeline for swh-web to test if it was feasible, I managed to setup the use of sorry-cypress but could not run multiple cypress in parallel as Jenkins pipelines forbid nested parallel stages.
So this would require setting up a dedicated pipeline build job for cypress tests as currently Python and cypress tests are executed in parallel in a single job.

Fortunately, it exists a workaround to allow nested parallel stages.

So I took a new stab at this and ended up with an updated Jenkins pipeline enabling to run cypress tests in parallel using 4 runners.

Using that approach, I observe a 2x speedup on overall swh-web tests execution, which is great !

Diffs for the result of that work are incoming.

anlambert added a revision: D8326: Modify some configuration to allow running cypress tests in parallel.Aug 26 2022, 5:20 PM

anlambert added a revision: D8328: templates/cypress: Run tests in parallel using sorry-cypress.Aug 26 2022, 5:28 PM

anlambert added a commit: rDWAPPS4b83f99981a4: cypress/add-forge-now-request-create: Add missing wait instructions.Aug 29 2022, 5:07 PM

anlambert added a commit: rDWAPPSb7be6a88dce7: cypress/origin-save: Add wait instructions to fix flaky tests.

anlambert added a commit: rDWAPPS4a8de541123b: cypress.config.js: Add projectId.

anlambert added a commit: rDWAPPS3adcbde7f785: cypress/admin: Add missing intercept in test implementation.

anlambert added a commit: rDWAPPScb03344f8a58: package.json: Add cy2 development dependency.

anlambert added a commit: rDWAPPS61b575db215d: sqlite: Use multiple database files when running parallel cypress tests.