Differential D8006

cvsclient: Retry pserver connection three times in case of failure
ClosedPublic
Actions

Authored by anlambert on Jun 17 2022, 5:35 PM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Commits

rDLDCVSb35a9769a035: cvsclient: Retry pserver connection three times in case of failure

Summary

Connection to an existing pserver might sometimes fail (on SourceForge
for instance), retrying the operation usally fixes the issue.

(swh) anlambert@carnavalet:/tmp/cvs_test$ swh loader -C ~/.config/swh/loader/cvs.yml run cvs pserver://anonymous@a.cvs.sourceforge.net/cvsroot/yazoo/help
INFO:swh.loader.cvs.loader.CvsLoader:Load origin 'pserver://anonymous@a.cvs.sourceforge.net/cvsroot/yazoo/help' with type 'cvs'
Request: BEGIN AUTH REQUEST
/cvsroot/yazoo
anonymous
A
END AUTH REQUEST


ERROR:swh.loader.cvs.loader.CvsLoader:Loading failure, updating to `not_found` status
Traceback (most recent call last):
  File "/home/anlambert/swh/swh-environment/swh-loader-core/swh/loader/core/loader.py", line 383, in load
    self.prepare()
  File "/home/anlambert/swh/swh-environment/swh-loader-cvs/swh/loader/cvs/loader.py", line 487, in prepare
    self.cvsclient = CVSClient(url)
  File "/home/anlambert/swh/swh-environment/swh-loader-cvs/swh/loader/cvs/cvsclient.py", line 227, in __init__
    self.connect_pserver(url.hostname, url.port, url.username, url.password)
  File "/home/anlambert/swh/swh-environment/swh-loader-cvs/swh/loader/cvs/cvsclient.py", line 115, in connect_pserver
    raise NotFound(
swh.loader.exception.NotFound: pserver authentication failed for a.cvs.sourceforge.net:2401: b'cvs pserver: cannot read /cvsroot/yazoo/CVSROOT/config: Operation not permitted\n'
{'status': 'uneventful'} for origin 'pserver://anonymous@a.cvs.sourceforge.net/cvsroot/yazoo/help'
(swh) anlambert@carnavalet:/tmp/cvs_test$ swh loader -C ~/.config/swh/loader/cvs.yml run cvs pserver://anonymous@a.cvs.sourceforge.net/cvsroot/yazoo/help
INFO:swh.loader.cvs.loader.CvsLoader:Load origin 'pserver://anonymous@a.cvs.sourceforge.net/cvsroot/yazoo/help' with type 'cvs'
Request: BEGIN AUTH REQUEST
/cvsroot/yazoo
anonymous
A
END AUTH REQUEST


{'status': 'eventful'} for origin 'pserver://anonymous@a.cvs.sourceforge.net/cvsroot/yazoo/help'

Diff Detail

Repository

rDLDCVS CVS Loader

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Jun 17 2022, 5:35 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJun 17 2022, 5:35 PM

Build is green

Patch application report for D8006 (id=28844)

Could not rebase; Attempt merge onto d52686be91...

Updating d52686b..b35a976
Fast-forward
 mypy.ini                            |  3 --
 swh/loader/cvs/cvsclient.py         | 62 +++++++++++++++++++++++--------------
 swh/loader/cvs/loader.py            |  9 +++---
 swh/loader/cvs/tests/test_loader.py | 23 +++++++++++++-
 4 files changed, 65 insertions(+), 32 deletions(-)

Changes applied before test

commit b35a9769a035eade4ac4a5acdc6fc8952bddb1cd
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Jun 17 17:33:03 2022 +0200

    cvsclient: Retry pserver connection three times in case of failure
    
    Connection to an existing pserver might sometimes fail (on SourceForge
    for instance), retrying the operation usally fixes the issue.

commit 089e2fd045232caa46e2e2d26f9ff6ba19f9d8d8
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Jun 17 16:55:38 2022 +0200

    cvsclient: Fix pserver error: "protocol error: <path> is not absolute"
    
    Some CVS servers (SourceForge and OSDN for instance) return an error if
    the path sent with the "Directory" pserver request is not absolute.
    
    So fix that issue to ensure loading of such CVS repositories.

commit e382aeb0526618ef5a7345551166ec46137fae93
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Fri Jun 17 16:36:34 2022 +0200

    cvsclient: Allow to connect to a pserver URL without password
    
    The CVS client was raising an error when trying to connect to such pserver
    URL: pserver://anonymous@cvs.example.org/cvsroot/project/module
    
    But numerous CVS pserver URLs that can be found in the wild (notably on
    SourceForge and OSDN) are in that form.
    
    So add support for such URL form in the CVS client.
    
    Also remove use of external dependency urllib3.util.parse_url and prefer
    to use urllib.parse.urlparse from standard Python library.

See https://jenkins.softwareheritage.org/job/DLDCVS/job/tests-on-diff/116/ for more details.

Harbormaster completed remote builds in B29929: Diff 28844.Jun 17 2022, 5:37 PM

anlambert requested review of this revision.Jun 17 2022, 5:37 PM