Page MenuHomeSoftware Heritage

add support for RCS keyword expansion over pserver protocol
ClosedPublic

Authored by stsp on Oct 30 2021, 11:47 AM.

Details

Summary

We can simply ask the CVS server to expand keywords for us, instead
of forcing binary file mode with the -kb option. The CVS repository
contains per-file keyword expansion defaults the server will use.
Files checked out by cvsclient.py should now match what a regular
CVS client would check out by default.

Add test cases which verify that we create the same snapshot ID
for a repository which uses the Id keyword in a file, regardless
of whether this repository is accessed via rsync or pserver.

Diff Detail

Repository
rDLDCVS CVS Loader
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24945
Build 38982: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 38981: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D6590 (id=23952)

Could not rebase; Attempt merge onto beb7fc8a02...

Updating beb7fc8..acc2601
Fast-forward
 swh/loader/cvs/cvsclient.py                     |  31 +++++---
 swh/loader/cvs/loader.py                        |   2 +-
 swh/loader/cvs/tests/data/greek-repository4.tgz | Bin 0 -> 12393 bytes
 swh/loader/cvs/tests/test_loader.py             |  92 +++++++++++++++++++++++-
 4 files changed, 114 insertions(+), 11 deletions(-)
 create mode 100644 swh/loader/cvs/tests/data/greek-repository4.tgz
Changes applied before test
commit acc2601ae97ca729e3987de7cbd3548120e5d5a1
Author: Stefan Sperling <stsp@stsp.name>
Date:   Sat Oct 30 11:31:27 2021 +0200

    add support for RCS keyword expansion over pserver protocol
    
    We can simply ask the CVS server to expand keywords for us, instead
    of forcing binary file mode with the -kb option. The CVS repository
    contains per-file keyword expansion defaults the server will use.
    Files checked out by cvsclient.py should now match what a regular
    CVS client would check out by default.
    
    Add test cases which verify that we create the same snapshot ID
    for a repository which uses the Id keyword in a file, regardless
    of whether this repository is accessed via rsync or pserver.

commit 2563ac0917db60491fd3d3218a47d16c97083d13
Author: Stefan Sperling <stsp@stsp.name>
Date:   Thu Oct 28 19:04:05 2021 +0200

    in cvs loader tests, use f-strings to build repository URLs
    
    Suggested by ardumont in D6566

See https://jenkins.softwareheritage.org/job/DLDCVS/job/tests-on-diff/43/ for more details.

stsp requested review of this revision.Oct 30 2021, 11:49 AM
vlorentz added inline comments.
swh/loader/cvs/cvsclient.py
309–320

Can you add a docstring?

417–429

did you mean to commit this?

removed cvsclient.py test code

Build is green

Patch application report for D6590 (id=24020)

Could not rebase; Attempt merge onto 7cffdf709f...

Updating 7cffdf7..f1254c8
Fast-forward
 swh/loader/cvs/cvs2gitdump/cvs2gitdump.py       | 105 ++++++++++++++----------
 swh/loader/cvs/cvsclient.py                     |  18 ++--
 swh/loader/cvs/loader.py                        |  44 ++++++----
 swh/loader/cvs/rcsparse.pyi                     |  22 ++++-
 swh/loader/cvs/rlog.py                          |  83 ++++++++++++-------
 swh/loader/cvs/tests/data/greek-repository4.tgz | Bin 0 -> 12393 bytes
 swh/loader/cvs/tests/test_loader.py             |  88 ++++++++++++++++++++
 7 files changed, 262 insertions(+), 98 deletions(-)
 create mode 100644 swh/loader/cvs/tests/data/greek-repository4.tgz
Changes applied before test
commit f1254c8a006549c32616be0d2f435bb3b5dde408
Author: Stefan Sperling <stsp@stsp.name>
Date:   Sat Oct 30 11:31:27 2021 +0200

    add support for RCS keyword expansion over pserver protocol
    
    We can simply ask the CVS server to expand keywords for us, instead
    of forcing binary file mode with the -kb option. The CVS repository
    contains per-file keyword expansion defaults the server will use.
    Files checked out by cvsclient.py should now match what a regular
    CVS client would check out by default.
    
    Add test cases which verify that we create the same snapshot ID
    for a repository which uses the Id keyword in a file, regardless
    of whether this repository is accessed via rsync or pserver.

commit 607884b8c30d3f2f1c90032e93cb00b63d211ce3
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Nov 3 17:21:29 2021 +0100

    Add type annotations
    
    Reviewers: stsp
    
    Differential Revision: https://forge.softwareheritage.org/D6598

See https://jenkins.softwareheritage.org/job/DLDCVS/job/tests-on-diff/51/ for more details.

add docstring to cvsclient's checkout() function

Build is green

Patch application report for D6590 (id=24023)

Rebasing onto 7e2d8d8996...

First, rewinding head to replay your work on top of it...
Applying: add support for RCS keyword expansion over pserver protocol
Using index info to reconstruct a base tree...
M	swh/loader/cvs/loader.py
Falling back to patching base and 3-way merge...
Auto-merging swh/loader/cvs/loader.py
Changes applied before test
commit a77fa8c36891842ef9ef15982f76b61a54566184
Author: Stefan Sperling <stsp@stsp.name>
Date:   Sat Oct 30 11:31:27 2021 +0200

    add support for RCS keyword expansion over pserver protocol
    
    We can simply ask the CVS server to expand keywords for us, instead
    of forcing binary file mode with the -kb option. The CVS repository
    contains per-file keyword expansion defaults the server will use.
    Files checked out by cvsclient.py should now match what a regular
    CVS client would check out by default.
    
    Add test cases which verify that we create the same snapshot ID
    for a repository which uses the Id keyword in a file, regardless
    of whether this repository is accessed via rsync or pserver.

See https://jenkins.softwareheritage.org/job/DLDCVS/job/tests-on-diff/52/ for more details.

This revision is now accepted and ready to land.Nov 9 2021, 11:42 AM

Build is green

Patch application report for D6590 (id=24070)

Rebasing onto 7e2d8d8996...

Current branch diff-target is up to date.
Changes applied before test
commit f52f0e4521320d4546ccce963e9d0ecc2e9cb17c
Author: Stefan Sperling <stsp@stsp.name>
Date:   Sat Oct 30 11:31:27 2021 +0200

    add support for RCS keyword expansion over pserver protocol
    
    We can simply ask the CVS server to expand keywords for us, instead
    of forcing binary file mode with the -kb option. The CVS repository
    contains per-file keyword expansion defaults the server will use.
    Files checked out by cvsclient.py should now match what a regular
    CVS client would check out by default.
    
    Add test cases which verify that we create the same snapshot ID
    for a repository which uses the Id keyword in a file, regardless
    of whether this repository is accessed via rsync or pserver.

See https://jenkins.softwareheritage.org/job/DLDCVS/job/tests-on-diff/57/ for more details.