Page MenuHomeSoftware Heritage
Differential D8509

gogs: Skip pages with error 500
ClosedPublic
Actions

Authored by KShivendu on Sep 20 2022, 1:16 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Commits
rDLSbd35d54398bf: gogs: Skip pages with error 500
Summary

Lister should skip the page if it encounters status code 500. It happens because of a few fatal repos (T4423)

Diff Detail

Repository
rDLS Listers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

KShivendu created this revision.Sep 20 2022, 1:16 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptSep 20 2022, 1:16 PM
swh-public-ci added a comment.Sep 20 2022, 1:21 PM

Build has FAILED

Patch application report for D8509 (id=30641)

Rebasing onto fa65f270ed...

First, rewinding head to replay your work on top of it...
Applying: gogs: Skip pages with error 500
Using index info to reconstruct a base tree...
M	swh/lister/gogs/lister.py
M	swh/lister/gogs/tests/test_lister.py
Falling back to patching base and 3-way merge...
Auto-merging swh/lister/gogs/tests/test_lister.py
Auto-merging swh/lister/gogs/lister.py
Changes applied before test
commit d8c1a83716cba600f1ff98c8f7efac46b19fdd5a
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500

Link to build: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/664/
See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/664/console

Harbormaster returned this revision to the author for changes because remote builds failed.Sep 20 2022, 1:21 PM
Harbormaster failed remote builds in B31620: Diff 30641!
KShivendu updated this revision to Diff 30642.Sep 20 2022, 2:03 PM
  • gogs: Skip pages with error 500
  • gitea: disable skipping on 500 status code
swh-public-ci added a comment.Sep 20 2022, 2:08 PM

Build is green

Patch application report for D8509 (id=30642)

Rebasing onto fa65f270ed...

Current branch diff-target is up to date.
Changes applied before test
commit 70772d6464bc98cde7d011a361e2600c02992fe9
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 17:31:50 2022 +0530

    gitea: disable skipping on 500 status code

commit 8101208c9a6e20497061b25c1501b5ee63e15bd2
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/665/ for more details.

Harbormaster completed remote builds in B31621: Diff 30642.Sep 20 2022, 2:08 PM
KShivendu requested review of this revision.Sep 20 2022, 2:08 PM
KShivendu added inline comments.Sep 20 2022, 2:10 PM
swh/lister/gitea/lister.py
45 ↗(On Diff #30642)

Should I get rid of this init and just make this a static property?

class GiteaLister:

skip_on_500 = False

def foo():
   ...
ardumont added a subscriber: ardumont.Sep 20 2022, 2:25 PM
ardumont added inline comments.
swh/lister/gitea/lister.py
45 ↗(On Diff #30642)

if that works, seems less verbose and simpler indeed, so yes ;)

45 ↗(On Diff #30642)

or just keep the same behavior for both lister (and drop all this altogether).

KShivendu updated this revision to Diff 30645.Sep 20 2022, 2:56 PM
  • gitea: skip on status code 500
swh-public-ci added a comment.Sep 20 2022, 3:01 PM

Build is green

Patch application report for D8509 (id=30645)

Rebasing onto fa65f270ed...

Current branch diff-target is up to date.
Changes applied before test
commit 7c092e215a56d797df893a30e3fa4fb40170230c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 17:31:50 2022 +0530

    gitea: skip on status code 500

commit 8101208c9a6e20497061b25c1501b5ee63e15bd2
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/666/ for more details.

Harbormaster completed remote builds in B31624: Diff 30645.Sep 20 2022, 3:01 PM
ardumont added inline comments.Sep 20 2022, 3:01 PM
swh/lister/gitea/tests/test_lister.py
142–144
swh/lister/gogs/lister.py
135

Pondering whether that's not all internal server errors which should be skipped... ?

ardumont added inline comments.Sep 20 2022, 3:27 PM
swh/lister/gogs/lister.py
135

15:14 <KShivendu> > Pondering whether that's not all internal server errors which should be skipped... ?
15:14 <KShivendu> as per https://developer.mozilla.org/en-US/docs/Web/HTTP/Status#server_error_responses, stuff like "Gateway timeout"
or "Service Unavailable" can also happen under 50x. it might be helpful to be aware of such failures so that we can retry later rather than
letting it fail silently. what do you think ardumont

yes, fine, let's keep your current version.

ardumont accepted this revision.Sep 20 2022, 3:28 PM
This revision is now accepted and ready to land.Sep 20 2022, 3:28 PM
KShivendu updated this revision to Diff 30648.Sep 20 2022, 3:37 PM
  • gitea: fix test name
  • squash and reword commits
swh-public-ci added a comment.Sep 20 2022, 3:42 PM

Build is green

Patch application report for D8509 (id=30648)

Rebasing onto fa65f270ed...

Current branch diff-target is up to date.
Changes applied before test
commit bd35d54398bf1d47c19bc1d8eb2e708d7e61bcd2
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Tue Sep 20 16:43:46 2022 +0530

    gogs: Skip pages with error 500
    
    This also affects the gitea lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/667/ for more details.

Harbormaster completed remote builds in B31627: Diff 30648.Sep 20 2022, 3:42 PM
Closed by commit rDLSbd35d54398bf: gogs: Skip pages with error 500 (authored by KShivendu). · Explain WhySep 20 2022, 3:43 PM
This revision was automatically updated to reflect the committed changes.
KShivendu added a commit: rDLSbd35d54398bf: gogs: Skip pages with error 500.

Revision Contents
Changeset List

PathSize
swh/
lister/
gitea/
tests/
31 lines
gogs/
43 lines
tests/
16 lines

Diff 30649

View Options

swh/lister/gitea/tests/test_lister.py

Loading...
View Options

swh/lister/gogs/lister.py

Loading...
View Options

swh/lister/gogs/tests/test_lister.py

Loading...
About · Developer information · Legal