Page MenuHomeSoftware Heritage
Differential D5761

cypress.json: Activate test retries in run mode
ClosedPublic
Actions

Authored by anlambert on May 20 2021, 1:57 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T3331: Fix flaky cypress tests
Commits
rDWAPPS7506c2d4e772: cypress.json: Activate test retries in run mode
Summary

A failing test will be executed at most five times.

Let's hope it will reduce the number of flaky tests on Jenkins.

Related to T3331

Diff Detail

Repository
rDWAPPS Web applications
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.May 20 2021, 1:57 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptMay 20 2021, 1:57 PM
swh-public-ci added a comment.May 20 2021, 2:13 PM

Build has FAILED

Patch application report for D5761 (id=20593)

Rebasing onto 5adccbb354...

Current branch diff-target is up to date.
Changes applied before test
commit a0df9a0f1fcdd76fd0127a1dae55730c324ee8ed
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu May 20 13:56:41 2021 +0200

    cypress.json: Activate test retries in run mode
    
    A failing test will be executed at most three times.
    
    Let's hope it will reduce the number of flaky tests on Jenkins.
    
    Related to T3331

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/836/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/836/console

Harbormaster returned this revision to the author for changes because remote builds failed.May 20 2021, 2:13 PM
Harbormaster failed remote builds in B21558: Diff 20593!
anlambert updated this revision to Diff 20594.May 20 2021, 2:17 PM

Increase retries to 4

anlambert added a comment.EditedMay 20 2021, 2:18 PM
In D5761#146517, @swh-public-ci wrote:

Build has FAILED

Patch application report for D5761 (id=20593)

Rebasing onto 5adccbb354...

Current branch diff-target is up to date.
Changes applied before test
commit a0df9a0f1fcdd76fd0127a1dae55730c324ee8ed
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu May 20 13:56:41 2021 +0200

    cypress.json: Activate test retries in run mode
    
    A failing test will be executed at most three times.
    
    Let's hope it will reduce the number of flaky tests on Jenkins.
    
    Related to T3331

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/836/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/836/console

Sigh... Let's try to increase the number of retries and launch a couple of subsequent builds to test.

swh-public-ci added a comment.May 20 2021, 2:26 PM

Build is green

Patch application report for D5761 (id=20594)

Rebasing onto 5adccbb354...

Current branch diff-target is up to date.
Changes applied before test
commit 36524f0d5413d49e3fca6e03ef015b9d0eb7a463
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu May 20 13:56:41 2021 +0200

    cypress.json: Activate test retries in run mode
    
    A failing test will be executed at most five times.
    
    Let's hope it will reduce the number of flaky tests on Jenkins.
    
    Related to T3331

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/837/ for more details.

Harbormaster completed remote builds in B21559: Diff 20594.May 20 2021, 2:26 PM
anlambert requested review of this revision.May 20 2021, 2:26 PM
swh-public-ci added a comment.May 20 2021, 2:38 PM

Build is green

Patch application report for D5761 (id=20594)

Rebasing onto 5adccbb354...

Current branch diff-target is up to date.
Changes applied before test
commit 36524f0d5413d49e3fca6e03ef015b9d0eb7a463
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu May 20 13:56:41 2021 +0200

    cypress.json: Activate test retries in run mode
    
    A failing test will be executed at most five times.
    
    Let's hope it will reduce the number of flaky tests on Jenkins.
    
    Related to T3331

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/838/ for more details.

swh-public-ci added a comment.May 20 2021, 2:49 PM

Build is green

Patch application report for D5761 (id=20594)

Rebasing onto 5adccbb354...

Current branch diff-target is up to date.
Changes applied before test
commit 36524f0d5413d49e3fca6e03ef015b9d0eb7a463
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu May 20 13:56:41 2021 +0200

    cypress.json: Activate test retries in run mode
    
    A failing test will be executed at most five times.
    
    Let's hope it will reduce the number of flaky tests on Jenkins.
    
    Related to T3331

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/839/ for more details.

anlambert edited the summary of this revision. (Show Details)May 20 2021, 2:52 PM
swh-public-ci added a comment.May 20 2021, 3:00 PM

Build is green

Patch application report for D5761 (id=20594)

Rebasing onto 5adccbb354...

Current branch diff-target is up to date.
Changes applied before test
commit 36524f0d5413d49e3fca6e03ef015b9d0eb7a463
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu May 20 13:56:41 2021 +0200

    cypress.json: Activate test retries in run mode
    
    A failing test will be executed at most five times.
    
    Let's hope it will reduce the number of flaky tests on Jenkins.
    
    Related to T3331

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/840/ for more details.

anlambert added a comment.May 20 2021, 3:02 PM

Four CI pipeline executions in a row and no flaky cypress tests when setting retries to 4. I guess we can land this then.

anlambert updated this revision to Diff 20597.May 20 2021, 3:04 PM

Update commit message

swh-public-ci added a comment.May 20 2021, 3:14 PM

Build is green

Patch application report for D5761 (id=20597)

Rebasing onto 5adccbb354...

Current branch diff-target is up to date.
Changes applied before test
commit 7506c2d4e77247da1b6563639da5fd8bef388099
Author: Antoine Lambert <antoine.lambert@inria.fr>
Date:   Thu May 20 13:56:41 2021 +0200

    cypress.json: Activate test retries in run mode
    
    A failing test will be executed at most five times.
    
    It enables to significantly reduce the number of flaky tests on Jenkins.
    
    Closes T3331

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/841/ for more details.

Harbormaster completed remote builds in B21562: Diff 20597.May 20 2021, 3:15 PM
anlambert added a comment.May 20 2021, 3:15 PM

Five CI pipeline executions in a row and no flaky cypress tests \o/

ardumont accepted this revision.May 20 2021, 3:25 PM
ardumont added a subscriber: ardumont.

ok then, let's try that ;)

This revision is now accepted and ready to land.May 20 2021, 3:25 PM
Closed by commit rDWAPPS7506c2d4e772: cypress.json: Activate test retries in run mode (authored by anlambert). · Explain WhyMay 20 2021, 3:27 PM
This revision was automatically updated to reflect the committed changes.
anlambert added a commit: rDWAPPS7506c2d4e772: cypress.json: Activate test retries in run mode.

Revision Contents
Changeset List

PathSize
3 lines

Diff 20600

View Options

cypress.json

Loading...
About · Developer information · Legal