Page MenuHomeSoftware Heritage

gogs: Skip pages with error 500
ClosedPublic

Authored by KShivendu on Tue, Sep 20, 1:16 PM.

Details

Summary

Lister should skip the page if it encounters status code 500. It happens because of a few fatal repos (T4423)

Diff Detail

Repository
rDLS Listers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build has FAILED

Patch application report for D8509 (id=30641)

Rebasing onto fa65f270ed...

First, rewinding head to replay your work on top of it...
Applying: gogs: Skip pages with error 500
Using index info to reconstruct a base tree...
M	swh/lister/gogs/lister.py
M	swh/lister/gogs/tests/test_lister.py
Falling back to patching base and 3-way merge...
Auto-merging swh/lister/gogs/tests/test_lister.py
Auto-merging swh/lister/gogs/lister.py
Changes applied before test
commit d8c1a83716cba600f1ff98c8f7efac46b19fdd5a
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/664/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/664/console

Harbormaster returned this revision to the author for changes because remote builds failed.Tue, Sep 20, 1:21 PM
Harbormaster failed remote builds in B31620: Diff 30641!
  • gogs: Skip pages with error 500
  • gitea: disable skipping on 500 status code

Build is green

Patch application report for D8509 (id=30642)

Rebasing onto fa65f270ed...

Current branch diff-target is up to date.
Changes applied before test
commit 70772d6464bc98cde7d011a361e2600c02992fe9
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 17:31:50 2022 +0530

    gitea: disable skipping on 500 status code

commit 8101208c9a6e20497061b25c1501b5ee63e15bd2
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/665/ for more details.

swh/lister/gitea/lister.py
45 ↗(On Diff #30642)

Should I get rid of this init and just make this a static property?

class GiteaLister:

skip_on_500 = False

def foo():
   ...
ardumont added inline comments.
swh/lister/gitea/lister.py
45 ↗(On Diff #30642)

if that works, seems less verbose and simpler indeed, so yes ;)

45 ↗(On Diff #30642)

or just keep the same behavior for both lister (and drop all this altogether).

  • gitea: skip on status code 500

Build is green

Patch application report for D8509 (id=30645)

Rebasing onto fa65f270ed...

Current branch diff-target is up to date.
Changes applied before test
commit 7c092e215a56d797df893a30e3fa4fb40170230c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 17:31:50 2022 +0530

    gitea: skip on status code 500

commit 8101208c9a6e20497061b25c1501b5ee63e15bd2
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/666/ for more details.

swh/lister/gitea/tests/test_lister.py
142–144
swh/lister/gogs/lister.py
135

Pondering whether that's not all internal server errors which should be skipped... ?

swh/lister/gogs/lister.py
135

15:14 <KShivendu> > Pondering whether that's not all internal server errors which should be skipped... ?
15:14 <KShivendu> as per https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#server_error_responses, stuff like "Gateway timeout"
or "Service Unavailable" can also happen under 50x. it might be helpful to be aware of such failures so that we can retry later rather than
letting it fail silently. what do you think ardumont

yes, fine, let's keep your current version.

This revision is now accepted and ready to land.Tue, Sep 20, 3:28 PM
  • gitea: fix test name
  • squash and reword commits

Build is green

Patch application report for D8509 (id=30648)

Rebasing onto fa65f270ed...

Current branch diff-target is up to date.
Changes applied before test
commit bd35d54398bf1d47c19bc1d8eb2e708d7e61bcd2
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500
    
    This also affects the gitea lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/667/ for more details.

This revision was automatically updated to reflect the committed changes.