Page MenuHomeSoftware Heritage
Create Task
Maniphest T2845

Improve Subversion loader and develop CVS loader
Closed, MigratedEdits Locked
Actions

Description

The proposal got approved and Stefan Sperling is going to work on this within the 2021
year.

This is the main task tracking the design, development, deployment and communication
work efforts.

Communication related, here is draft text for the announcement link.

Revisions and Commits

rCDFJ Dockerfiles for Jenkins
D6278rCDFJe3ee83908d2b Update container to install the cvs dependency
rDDOC Development documentation
D5966rDDOCc803ad53f80b Update documentation entry point with the new loader cvs module
rDLDCVS CVS Loader
D5988rDLDCVS970e86ad27dd make 'black' ignore cvsclient.py; forced reformatting is unreadable
D5988rDLDCVS56338c735119 remove rcsparse testmodule; pointless test which tox tried to run
D5988rDLDCVS3e5f10269f51 tell 'black' to ignore rcsparse and cvs2gitdump sources
D5988rDLDCVS26ab35aa07d1 fix pytest arguments in tox.ini
D5988rDLDCVSea4c7fa15a3e rcsparse.pyi: 'black' prefers empty line after imports
D5988rDLDCVS71157f208bf0 tell flake8 to ignore invalid escape sequences in regex
D5988rDLDCVSc5e647809673 remove cvs2svndump; it fails linter checks and we don't use it
D5988rDLDCVS9965c6b897ae reformat cvsclient.py to appease flake8
D5988rDLDCVS87da9cee4c6f reformat test_loader.py to appease flake8
D5988rDLDCVS26d6891c6ecf fix typo in variable name; found by flake8
D5988rDLDCVSacc88adda271 also add .* to the default exclude list
D5988rDLDCVS91ca5ca35414 avoid use of undefined module vclib; found by flake8
D5988rDLDCVS9774173fc66f fix reference to subprocess.TimeoutExpired; found by flake8
D5988rDLDCVS1c7635b9b3e8 wrap overlong lines in rlog.py to appease flake8
D5988rDLDCVS73ad331915e3 add a comment which documents RCS/CVS branch numbers
D5988rDLDCVS1dd3f43e9e46 tell flake8 to ignore the 'build' directory
D5988rDLDCVS1080eb83c318 add a very simple .pyi for rcsparse to satisfy mypy
D5988rDLDCVSd6c1817d39c0 document CVS path encoding issues
D5988rDLDCVSb80aa0c27997 make a line of code more readable
D5988rDLDCVS871042808269 add an assert to avoid a None type error from mypy
D5988rDLDCVS52faf13b77f4 convert readme file from .md to .rst
D5988rDLDCVSfa9121756333 simplify use of os.makedirs(); from vlorentz
D5988rDLDCVSd5c46c5cb5f2 merge nested functions which yield into single functions
D5988rDLDCVS0fe0e68f29c2 acknowledge code derived from ViewVC in our README file
D5988rDLDCVS883aa39c8c93 fix incomplete docstring of swh_hash_data_per_cvs_changeset
D5988rDLDCVS1c0acc70f1f5 avoid pointless conversion to bytes() and use (x,) tuple idiom
D5988rDLDCVS2a4b87f9999f Remove an unused import.
D5988rDLDCVS0872e9e61663 exclude third-party sources via mypy.ini, not pre-commit conf
D5988rDLDCVSb2fb227a2cf6 fix format string error found by pre-commit hook
D5988rDLDCVS50ae8b09e12e fix 'ssh' protocol support
D5988rDLDCVSff50851e8183 exclude third-party sources from flake8 checks
D5988rDLDCVSd125c5c7b0f0 the pre-commit hook complained about an unused rcsfile variable
D5988rDLDCVS4e65e78bf063 commit reformatting of rlog.py done by pre-commit hook
D5988rDLDCVS82e8d797cf0a wrap overlong lines
D5988rDLDCVS5e3904dd50ab committing reformatting which was performed by pre-commit hooks
D5988rDLDCVS0e1e757565ac exclude upstream python code from pre-commit checks
D5988rDLDCVSc479b0e26006 fix myphy errors on swh/loader/__init__.py
D5988rDLDCVS98082f20e7e5 urllib doesn't have stubs for mypy
D5988rDLDCVS6548f6ca45dd commit reformatting performed by pre-commit hook on setup.py
D5988rDLDCVS07ed9fc946b1 update README
D5988rDLDCVSc04bb816c7f8 remove unused keyword parameter to fix loading of the cvs loader
D5988rDLDCVSc721a06cb62b remove unused imports
D5988rDLDCVSba728189a3c5 implement support for import via cvs pserver protocol
D5988rDLDCVS1eaaa52b054c simply assign to empty lists in order to clear per-changeset data
D5988rDLDCVS4ddc08250622 switch log level of an informative progress message from debug to info
D5988rDLDCVSe7ba43fd93cb add test case doing an incremental visit
D5988rDLDCVS250bb9dbba27 avoid double-parsing of rcsfiles while processing a changeset
D5988rDLDCVS092b5263f936 replace global self.rcs variable with use of local variables
D5988rDLDCVS551a12e24cb8 drop support for using previous snapshots as base for new ones
D5988rDLDCVS76c249b4a8c3 copy over the pre_cleanup() handler from the SVN loader
D5988rDLDCVS89d7b0fdd2ef cvs2gitdump: avoid parsing rcs files inside expand_keyword()
D5988rDLDCVS0424433458bd remove unused CvsLoader class members and constructor arguments
D5988rDLDCVS71f22dbe195f change logging level of per-revision info from DEBUG to INFO
D5988rDLDCVScf5cf4594d05 link revisions to their parents and add another small test
D5988rDLDCVSed460a3280fe do not call self.storage.revision_get() more often than necessary
D5988rDLDCVScdbcdcc92e61 enable check_snapshot() in test_loader_cvs_visit()
D5988rDLDCVSedc28c215955 fix rsync URL processing in fetch_cvs_repo_with_rsync()
D5988rDLDCVS5097bf8eaab1 use an iterator to process swh revisions; reduces memory usage
D5988rDLDCVSae77fa161841 test two consecutive visits
D5988rDLDCVS496ff263546a the trivial first visit test is passing now
D5988rDLDCVS358a05d59956 add converted CVS revisions to storage
D5988rDLDCVS6a2f6e50f61e ignore ENOENT when removing files
D5988rDLDCVS59a7ff18d603 document fetch_data() method
D5988rDLDCVS49ce9a24d6f9 create subdirectories in the work tree
D5988rDLDCVSaf30f67f7403 populate a work tree with files checked out from the repository
D5988rDLDCVS15252a6ce822 force our local rcsparse dependency; upstream does not yet support py3
D5988rDLDCVS799eea5a7b68 get fetch_data working
D5988rDLDCVS605d8f5862b8 get the 'prepare' step working
D5988rDLDCVS757ce82f02eb remove bogus import
D5988rDLDCVS28cad58a0f32 add stub for an initial test
D5988rDLDCVS9dec8a2e4fdc rcsparse: add missing allocation failure check in parsetoken()
D5988rDLDCVSaaec7d33bf74 remove the 'foo' module and references to it
D5988rDLDCVS9e3170f81c2b rcsparse python3 support patches from OpenBSD ports
D5988rDLDCVSecf402cf6249 compile rcsparse extension
D5988rDLDCVS75fe9f5ea285 Add 'swh/loader/cvs/rcsparse/'
D5988rDLDCVSbb4362177743 Add 'swh/loader/cvs/cvs2gitdump/'
D5988rDLDCVS65995f14b4c6 initial CVS loader stub

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
rdicosmo updated the task description. (Show Details)Dec 2 2020, 4:38 PM
rdicosmo changed the edit policy from "All Users" to "Unknown Object (Project) (Project)".
rdicosmo moved this task from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.Mar 12 2021, 11:23 AM
ardumont mentioned this in rCJSWHede4a65bc9e1: Reference swh-loader-cvs project.Jul 5 2021, 11:24 AM
ardumont mentioned this in rDENVabd68bc60744: Reference swh-loader-cvs repository.Jul 5 2021, 11:34 AM
ardumont added a revision: D5966: Update documentation entry point with the new loader cvs module.Jul 5 2021, 11:46 AM
ardumont added a commit: rDDOCc803ad53f80b: Update documentation entry point with the new loader cvs module.Jul 5 2021, 11:50 AM
ardumont added a revision: D6278: Update container to install the cvs dependency.Sep 16 2021, 11:23 AM
ardumont added a commit: rCDFJe3ee83908d2b: Update container to install the cvs dependency.Sep 16 2021, 11:47 AM
ardumont added a revision: D5988: initial CVS loader stub.Sep 16 2021, 11:53 AM
stsp added a commit: rDLDCVS65995f14b4c6: initial CVS loader stub.Sep 17 2021, 2:49 PM
stsp added a commit: rDLDCVSbb4362177743: Add 'swh/loader/cvs/cvs2gitdump/'.
stsp added a commit: rDLDCVS75fe9f5ea285: Add 'swh/loader/cvs/rcsparse/'.
stsp added a commit: rDLDCVSecf402cf6249: compile rcsparse extension.
stsp added a commit: rDLDCVS9e3170f81c2b: rcsparse python3 support patches from OpenBSD ports.
stsp added a commit: rDLDCVSaaec7d33bf74: remove the 'foo' module and references to it.
stsp added a commit: rDLDCVS9dec8a2e4fdc: rcsparse: add missing allocation failure check in parsetoken().
stsp added a commit: rDLDCVS28cad58a0f32: add stub for an initial test.
stsp added a commit: rDLDCVS757ce82f02eb: remove bogus import.
stsp added a commit: rDLDCVS605d8f5862b8: get the 'prepare' step working.
stsp added a commit: rDLDCVS799eea5a7b68: get fetch_data working.
stsp added a commit: rDLDCVS15252a6ce822: force our local rcsparse dependency; upstream does not yet support py3.
stsp added a commit: rDLDCVSaf30f67f7403: populate a work tree with files checked out from the repository.
stsp added a commit: rDLDCVS49ce9a24d6f9: create subdirectories in the work tree.
stsp added a commit: rDLDCVS59a7ff18d603: document fetch_data() method.
stsp added a commit: rDLDCVS6a2f6e50f61e: ignore ENOENT when removing files.
stsp added a commit: rDLDCVS358a05d59956: add converted CVS revisions to storage.
stsp added a commit: rDLDCVS496ff263546a: the trivial first visit test is passing now.
stsp added a commit: rDLDCVSae77fa161841: test two consecutive visits.
stsp added a commit: rDLDCVSed460a3280fe: do not call self.storage.revision_get() more often than necessary.
stsp added a commit: rDLDCVScdbcdcc92e61: enable check_snapshot() in test_loader_cvs_visit().
stsp added a commit: rDLDCVSedc28c215955: fix rsync URL processing in fetch_cvs_repo_with_rsync().
stsp added a commit: rDLDCVS5097bf8eaab1: use an iterator to process swh revisions; reduces memory usage.
stsp added a commit: rDLDCVS71f22dbe195f: change logging level of per-revision info from DEBUG to INFO.
stsp added a commit: rDLDCVScf5cf4594d05: link revisions to their parents and add another small test.
stsp added a commit: rDLDCVS0424433458bd: remove unused CvsLoader class members and constructor arguments.
stsp added a commit: rDLDCVS551a12e24cb8: drop support for using previous snapshots as base for new ones.
stsp added a commit: rDLDCVS092b5263f936: replace global self.rcs variable with use of local variables.
stsp added a commit: rDLDCVS89d7b0fdd2ef: cvs2gitdump: avoid parsing rcs files inside expand_keyword().
stsp added a commit: rDLDCVS76c249b4a8c3: copy over the pre_cleanup() handler from the SVN loader.
stsp added a commit: rDLDCVS250bb9dbba27: avoid double-parsing of rcsfiles while processing a changeset.
stsp added a commit: rDLDCVSc721a06cb62b: remove unused imports.
stsp added a commit: rDLDCVSe7ba43fd93cb: add test case doing an incremental visit.
stsp added a commit: rDLDCVS1eaaa52b054c: simply assign to empty lists in order to clear per-changeset data.
stsp added a commit: rDLDCVS4ddc08250622: switch log level of an informative progress message from debug to info.
stsp added a commit: rDLDCVSba728189a3c5: implement support for import via cvs pserver protocol.
stsp added a commit: rDLDCVSc04bb816c7f8: remove unused keyword parameter to fix loading of the cvs loader.
stsp added a commit: rDLDCVS07ed9fc946b1: update README.
stsp added a commit: rDLDCVS5e3904dd50ab: committing reformatting which was performed by pre-commit hooks.
stsp added a commit: rDLDCVS0e1e757565ac: exclude upstream python code from pre-commit checks.
stsp added a commit: rDLDCVS98082f20e7e5: urllib doesn't have stubs for mypy.
stsp added a commit: rDLDCVSc479b0e26006: fix myphy errors on swh/loader/__init__.py.
stsp added a commit: rDLDCVS82e8d797cf0a: wrap overlong lines.
stsp added a commit: rDLDCVS6548f6ca45dd: commit reformatting performed by pre-commit hook on setup.py.
stsp added a commit: rDLDCVSb2fb227a2cf6: fix format string error found by pre-commit hook.
stsp added a commit: rDLDCVSd125c5c7b0f0: the pre-commit hook complained about an unused rcsfile variable.
stsp added a commit: rDLDCVS0872e9e61663: exclude third-party sources via mypy.ini, not pre-commit conf.
stsp added a commit: rDLDCVSff50851e8183: exclude third-party sources from flake8 checks.
stsp added a commit: rDLDCVS50ae8b09e12e: fix 'ssh' protocol support.
stsp added a commit: rDLDCVS4e65e78bf063: commit reformatting of rlog.py done by pre-commit hook.
stsp added a commit: rDLDCVS2a4b87f9999f: Remove an unused import..
stsp added a commit: rDLDCVS0fe0e68f29c2: acknowledge code derived from ViewVC in our README file.
stsp added a commit: rDLDCVS52faf13b77f4: convert readme file from .md to .rst.
stsp added a commit: rDLDCVSfa9121756333: simplify use of os.makedirs(); from vlorentz.
stsp added a commit: rDLDCVSd5c46c5cb5f2: merge nested functions which yield into single functions.
stsp added a commit: rDLDCVS1c0acc70f1f5: avoid pointless conversion to bytes() and use (x,) tuple idiom.
stsp added a commit: rDLDCVS883aa39c8c93: fix incomplete docstring of swh_hash_data_per_cvs_changeset.
stsp added a commit: rDLDCVS1080eb83c318: add a very simple .pyi for rcsparse to satisfy mypy.
stsp added a commit: rDLDCVS871042808269: add an assert to avoid a None type error from mypy.
stsp added a commit: rDLDCVS1dd3f43e9e46: tell flake8 to ignore the 'build' directory.
stsp added a commit: rDLDCVSb80aa0c27997: make a line of code more readable.
stsp added a commit: rDLDCVSd6c1817d39c0: document CVS path encoding issues.
stsp added a commit: rDLDCVS73ad331915e3: add a comment which documents RCS/CVS branch numbers.
stsp added a commit: rDLDCVS1c7635b9b3e8: wrap overlong lines in rlog.py to appease flake8.
stsp added a commit: rDLDCVSacc88adda271: also add .* to the default exclude list.
stsp added a commit: rDLDCVS87da9cee4c6f: reformat test_loader.py to appease flake8.
stsp added a commit: rDLDCVS9965c6b897ae: reformat cvsclient.py to appease flake8.
stsp added a commit: rDLDCVS91ca5ca35414: avoid use of undefined module vclib; found by flake8.
stsp added a commit: rDLDCVS9774173fc66f: fix reference to subprocess.TimeoutExpired; found by flake8.
stsp added a commit: rDLDCVS26d6891c6ecf: fix typo in variable name; found by flake8.
stsp added a commit: rDLDCVS26ab35aa07d1: fix pytest arguments in tox.ini.
stsp added a commit: rDLDCVS71157f208bf0: tell flake8 to ignore invalid escape sequences in regex.
stsp added a commit: rDLDCVSc5e647809673: remove cvs2svndump; it fails linter checks and we don't use it.
stsp added a commit: rDLDCVS3e5f10269f51: tell 'black' to ignore rcsparse and cvs2gitdump sources.
stsp added a commit: rDLDCVSea4c7fa15a3e: rcsparse.pyi: 'black' prefers empty line after imports.
stsp added a commit: rDLDCVS970e86ad27dd: make 'black' ignore cvsclient.py; forced reformatting is unreadable.
stsp added a commit: rDLDCVS56338c735119: remove rcsparse testmodule; pointless test which tox tried to run.
zack changed the edit policy from "Unknown Object (Project) (Project)" to "All Users".Oct 26 2021, 11:44 AM
ardumont removed revisions: D5988: initial CVS loader stub, D6278: Update container to install the cvs dependency, D5966: Update documentation entry point with the new loader cvs module.Oct 26 2021, 11:56 AM
ardumont renamed this task from Subversion and CVS to Improve Subversion loader and develop CVS loader.Oct 26 2021, 12:19 PM
ardumont edited projects, added Archive coverage; removed Unknown Object (Project).
ardumont shifted this object from the Restricted Space space to the S1 Public space.
ardumont updated the task description. (Show Details)Oct 27 2021, 10:05 AM
gitlab-migration closed subtask T3691: Implement CVS loader as Migrated.Jan 8 2023, 5:03 PM
gitlab-migration closed subtask T3692: Stabilize svn loader implementation as Migrated.
gitlab-migration closed this task as Migrated.Jan 8 2023, 10:22 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal