GitLab API can return errors 500 when listing projects
(see https://gitlab.com/gitlab-org/gitlab/-/issues/262629).
To avoid ending the listing prematurely, skip buggy URLs and move
to next pages.
Related to T3442
Depends on D6021
Details
Details
- Reviewers
ardumont - Group Reviewers
Reviewers - Maniphest Tasks
- T3442: Make GitLab lister more robust to HTTP errors
- Commits
- rDLSe904f4760ef1: gitlab: Handle HTTP status code 500 when listing projects
Diff Detail
Diff Detail
- Repository
- rDLS Listers
- Branch
- gitlab-mitigate-error-500
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 22729 Build 35447: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 35446: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D6022 (id=21762)
Could not rebase; Attempt merge onto 73f85c0b8a...
Updating 73f85c0..35ced44 Fast-forward swh/lister/gitlab/lister.py | 23 ++++++++++++++++++++++- swh/lister/gitlab/tests/test_lister.py | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 1 deletion(-)
Changes applied before test
commit 35ced44b52a0b67f673fe6c6d6f8c698401d3e72
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Fri Jul 23 14:33:15 2021 +0200
gitlab: Handle HTTP status code 500 when listing projects
GitLab API can return errors 500 when listing projects
(see https://gitlab.com/gitlab-org/gitlab/-/issues/262629).
To avoid ending the listing prematurely, skip buggy URLs and move
to next pages.
Related to T3442
commit 52c315015543340c9647174f206aabcfa438feb4
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Fri Jul 23 13:59:52 2021 +0200
gitlab: Update requests query parameters
Increase number of origins per page to the maximum value allowed
by GitLab API (100) to send less requests.
Ask for simple responses to reduce size of JSON data.See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/335/ for more details.
Comment Actions
Build is green
Patch application report for D6022 (id=21763)
Could not rebase; Attempt merge onto 73f85c0b8a...
Updating 73f85c0..e904f47 Fast-forward swh/lister/gitlab/lister.py | 23 ++++++++++++++++++++++- swh/lister/gitlab/tests/test_lister.py | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 1 deletion(-)
Changes applied before test
commit e904f4760ef10a99fc5c89bfb5cd4e28e6efbd95
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Fri Jul 23 14:33:15 2021 +0200
gitlab: Handle HTTP status code 500 when listing projects
GitLab API can return errors 500 when listing projects
(see https://gitlab.com/gitlab-org/gitlab/-/issues/262629).
To avoid ending the listing prematurely, skip buggy URLs and move
to next pages.
Related to T3442
commit 52c315015543340c9647174f206aabcfa438feb4
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date: Fri Jul 23 13:59:52 2021 +0200
gitlab: Update requests query parameters
Increase number of origins per page to the maximum value allowed
by GitLab API (100) to send less requests.
Ask for simple responses to reduce size of JSON data.See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/336/ for more details.