Differential D6361

mercurial: Rename from_disk module to the main and official loader
ClosedPublic
Actions

Authored by ardumont on Sep 28 2021, 11:24 AM.

Details

Reviewers

vlorentz

Group Reviewers

Reviewers

Maniphest Tasks

T3612: Clean up mercurial loader code

Commits

rDLDHGedb7ac60a5fd: mercurial: Rename from_disk module to the main and official loader

Summary

Note: This impacts production:

the task must be renamed in the scheduler backend
the worker configuration must be changed to drop the no longer used names
the docker/kubernetes configuration must be adapted as well

I'll attend to those once i've landed/packaged this.

Related to T3612
Depends on D6360

Test Plan

tox

Diff Detail

Repository

rDLDHG Mercurial loader

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 24034
Build 37499: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 37498: arc lint + arc unit

Event Timeline

ardumont created this revision.Sep 28 2021, 11:24 AM

Build is green

Patch application report for D6361 (id=23117)

Could not rebase; Attempt merge onto 395ea80518...

Updating 395ea80..2fa0575
Fast-forward
 README.md                                          |    8 +-
 conftest.py                                        |    3 +-
 setup.py                                           |    3 +-
 swh/loader/mercurial/__init__.py                   |   16 +-
 swh/loader/mercurial/bundle20_reader.py            |  621 -----------
 swh/loader/mercurial/chunked_reader.py             |  105 --
 swh/loader/mercurial/cli.py                        |   66 --
 swh/loader/mercurial/from_disk.py                  |  815 --------------
 swh/loader/mercurial/loader.py                     | 1142 +++++++++++---------
 swh/loader/mercurial/tasks.py                      |   13 +-
 swh/loader/mercurial/tasks_from_disk.py            |   43 -
 swh/loader/mercurial/tests/test_from_disk.py       |  738 -------------
 swh/loader/mercurial/tests/test_loader.py          |  721 ++++++++----
 swh/loader/mercurial/tests/test_tasks.py           |    8 +-
 swh/loader/mercurial/tests/test_tasks_from_disk.py |   47 -
 15 files changed, 1160 insertions(+), 3189 deletions(-)
 delete mode 100644 swh/loader/mercurial/bundle20_reader.py
 delete mode 100644 swh/loader/mercurial/chunked_reader.py
 delete mode 100644 swh/loader/mercurial/cli.py
 delete mode 100644 swh/loader/mercurial/from_disk.py
 delete mode 100644 swh/loader/mercurial/tasks_from_disk.py
 delete mode 100644 swh/loader/mercurial/tests/test_from_disk.py
 delete mode 100644 swh/loader/mercurial/tests/test_tasks_from_disk.py

Changes applied before test

commit 2fa0575e0b1abddb21157d88acf059eb79b71038
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Sep 28 11:21:54 2021 +0200

    mercurial: Rename from_disk module to the main and official loader
    
    Related to T3612

commit c46acc30ec3dfa69a95a4ecdc14b4ebdf2b62bdd
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Sep 28 10:46:37 2021 +0200

    mercurial: Drop dead code
    
    Keep only the new mercurial loader
    
    Related to T3612

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/316/ for more details.

Harbormaster completed remote builds in B24034: Diff 23117.Sep 28 2021, 11:26 AM

ardumont requested review of this revision.Sep 28 2021, 11:26 AM

Fix missing old from_disk reference

Build is green

Patch application report for D6361 (id=23118)

Could not rebase; Attempt merge onto 395ea80518...

Updating 395ea80..a36c18b
Fast-forward
 README.md                                          |    8 +-
 conftest.py                                        |    3 +-
 setup.py                                           |    3 +-
 swh/loader/mercurial/__init__.py                   |   16 +-
 swh/loader/mercurial/bundle20_reader.py            |  621 -----------
 swh/loader/mercurial/chunked_reader.py             |  105 --
 swh/loader/mercurial/cli.py                        |   66 --
 swh/loader/mercurial/from_disk.py                  |  815 --------------
 swh/loader/mercurial/loader.py                     | 1142 +++++++++++---------
 swh/loader/mercurial/tasks.py                      |   13 +-
 swh/loader/mercurial/tasks_from_disk.py            |   43 -
 swh/loader/mercurial/tests/test_from_disk.py       |  738 -------------
 swh/loader/mercurial/tests/test_loader.py          |  721 ++++++++----
 swh/loader/mercurial/tests/test_tasks.py           |    8 +-
 swh/loader/mercurial/tests/test_tasks_from_disk.py |   47 -
 15 files changed, 1160 insertions(+), 3189 deletions(-)
 delete mode 100644 swh/loader/mercurial/bundle20_reader.py
 delete mode 100644 swh/loader/mercurial/chunked_reader.py
 delete mode 100644 swh/loader/mercurial/cli.py
 delete mode 100644 swh/loader/mercurial/from_disk.py
 delete mode 100644 swh/loader/mercurial/tasks_from_disk.py
 delete mode 100644 swh/loader/mercurial/tests/test_from_disk.py
 delete mode 100644 swh/loader/mercurial/tests/test_tasks_from_disk.py

Changes applied before test

commit a36c18b7c428c03644707ff2edc80584376e9b8a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Sep 28 11:21:54 2021 +0200

    mercurial: Rename from_disk module to the main and official loader
    
    Related to T3612

commit c46acc30ec3dfa69a95a4ecdc14b4ebdf2b62bdd
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Sep 28 10:46:37 2021 +0200

    mercurial: Drop dead code
    
    Keep only the new mercurial loader
    
    Related to T3612

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/317/ for more details.

Harbormaster completed remote builds in B24035: Diff 23118.Sep 28 2021, 11:30 AM

vlorentz accepted this revision.Sep 28 2021, 1:04 PM

vlorentz added a subscriber: vlorentz.

vlorentz added inline comments.

swh/loader/mercurial/loader.py
5

This revision is now accepted and ready to land.Sep 28 2021, 1:04 PM

Rebase and adapt docstring according to suggestion

Build is green

Patch application report for D6361 (id=23121)

Could not rebase; Attempt merge onto 395ea80518...

Updating 395ea80..edb7ac6
Fast-forward
 README.md                                          |    8 +-
 conftest.py                                        |    3 +-
 setup.py                                           |    3 +-
 swh/loader/mercurial/__init__.py                   |   16 +-
 swh/loader/mercurial/bundle20_reader.py            |  621 -----------
 swh/loader/mercurial/chunked_reader.py             |  105 --
 swh/loader/mercurial/cli.py                        |   66 --
 swh/loader/mercurial/from_disk.py                  |  815 --------------
 swh/loader/mercurial/loader.py                     | 1142 +++++++++++---------
 swh/loader/mercurial/tasks.py                      |   13 +-
 swh/loader/mercurial/tasks_from_disk.py            |   43 -
 swh/loader/mercurial/tests/test_from_disk.py       |  738 -------------
 swh/loader/mercurial/tests/test_loader.py          |  721 ++++++++----
 swh/loader/mercurial/tests/test_tasks.py           |    8 +-
 swh/loader/mercurial/tests/test_tasks_from_disk.py |   47 -
 15 files changed, 1160 insertions(+), 3189 deletions(-)
 delete mode 100644 swh/loader/mercurial/bundle20_reader.py
 delete mode 100644 swh/loader/mercurial/chunked_reader.py
 delete mode 100644 swh/loader/mercurial/cli.py
 delete mode 100644 swh/loader/mercurial/from_disk.py
 delete mode 100644 swh/loader/mercurial/tasks_from_disk.py
 delete mode 100644 swh/loader/mercurial/tests/test_from_disk.py
 delete mode 100644 swh/loader/mercurial/tests/test_tasks_from_disk.py

Changes applied before test

commit edb7ac60a5fdf4d9849ff4315a04644ceacf7534
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Sep 28 11:21:54 2021 +0200

    mercurial: Rename from_disk module to the main and official loader
    
    Related to T3612

commit 37382ae61dc70380d18b63cb21686a6070b66ad5
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Tue Sep 28 10:46:37 2021 +0200

    mercurial: Drop legacy loader
    
    Keep only the latest mercurial loader. In another commit, that loader will get renamed
    appropriately.
    
    Related to T3612

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/319/ for more details.

Harbormaster completed remote builds in B24038: Diff 23121.Sep 28 2021, 1:34 PM

Closed by commit rDLDHGedb7ac60a5fd: mercurial: Rename from_disk module to the main and official loader (authored by ardumont). · Explain WhySep 29 2021, 8:29 AM

This revision was automatically updated to reflect the committed changes.

ardumont added a commit: rDLDHGedb7ac60a5fd: mercurial: Rename from_disk module to the main and official loader.