Page MenuHomeSoftware Heritage

bzr: Allow cloning with breezy version without cmd_clone function
AbandonedPublicDraft

Authored by ardumont on Feb 9 2022, 2:31 PM.

Details

Reviewers
Alphare
Group Reviewers
Reviewers
Maniphest Tasks
T3915: Deploy Bzr loader to the staging environment
Summary

Without this, on stable node, the actual loader would not clone properly the bazar
repository. The function actually doing the cloning is defined for version 3.1
onward (on stable, we are running the 3.0).

Related to T3915

Test Plan

in the bzr_clone function defined (drop the code doing the import from new breezy
version, only keep the subprocess call and it's all green nonetheless).

That is:

$ grep -A17 "def bzr_clone" swh/loader/bzr/loader.py
def bzr_clone(origin_url: str, repo_directory: str) -> Callable:
    """Bazaar clone function"""
    # try:
    #     from breezy.builtins import cmd_clone
    #     closure_clone = partial(cmd_clone().run, origin_url, repo_directory)
    # except ImportError:
    #     # stable python3.breezy does not define the cmd_clone yet
    #     # fallback
    #     def closure_clone():
    #         from subprocess import run
    #         run(["bzr", "clone", origin_url, repo_directory])
    # stable python3.breezy does not define the cmd_clone yet
    # fallback
    def closure_clone():
        from subprocess import run
        run(["bzr", "clone", origin_url, repo_directory])

    return closure_clone
$ pytest --ff -x -s swh/loader/bzr/tests
============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.9.2, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /home/tony/work/inria/repo/swh/swh-environment/swh-loader-bzr, configfile: pytest.ini
plugins: redis-2.1.1, forked-1.3.0, asyncio-0.15.1, django-4.4.0, django-test-migrations-1.1.0, requests-mock-1.9.3, flask-1.2.0, celery-0.0.0, mock-3.6.1, hypothesis-6.21.0, postgresql-2.6.1, dash-2.0.0, xdist-2.3.0, swh.core-1.1.1, swh.journal-1.0.0
collected 14 items
run-last-failure: no previously failed tests, not deselecting items.

swh/loader/bzr/tests/test_loader.py   create_plpgsql_language
----------------------------
 language already installed
(1 row)

Created new control directory.
............
swh/loader/bzr/tests/test_tasks.py [2022-02-09 14:34:04,653: INFO/MainProcess] Connected to memory://guest@localhost//
[2022-02-09 14:34:05,661: INFO/MainProcess] Task celery.ping[9f1f8a6b-0460-448b-b57c-0839b1b3c624] received
[2022-02-09 14:34:05,663: INFO/MainProcess] Task celery.ping[9f1f8a6b-0460-448b-b57c-0839b1b3c624] succeeded in 0.0013701990028494038s: 'pong'
[2022-02-09 14:34:06,665: INFO/MainProcess] Task swh.loader.bzr.tasks.LoadBazaar[45864edb-90cd-4c48-ba5d-3e04c281e97e] received
[2022-02-09 14:34:06,672: INFO/MainProcess] Task swh.loader.bzr.tasks.LoadBazaar[45864edb-90cd-4c48-ba5d-3e04c281e97e] succeeded in 0.006103710002207663s: {'status': 'eventful'}
.

============================================================================================================== warnings summary ===============================================================================================================
swh/loader/bzr/tests/test_tasks.py::test_loader
  /home/tony/.virtualenvs/swh/lib/python3.9/site-packages/mercurial/utils/resourceutil.py:12: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================================================================================================== 14 passed, 1 warning in 7.94s ========================================================================================================

tox

Diff Detail

Repository
rDLDBZR BZR loader
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26693
Build 41741: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 41740: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D7132 (id=25851)

Rebasing onto 078fb50296...

Current branch diff-target is up to date.
Changes applied before test
commit 6173626db6b6651b3ee522f8428aed02c2a5c753
Author: Antoine R. Dumont (@ardumont) <antoine.romain.dumont@gmail.com>
Date:   Wed Feb 9 14:28:12 2022 +0100

    bzr: Allow cloning with breezy version without cmd_clone function
    
    Without this, on stable node, the actual loader would not clone properly the bazar
    repository. The function actually doing the cloning is defined for version 3.1
    onward (on stable, we are running the 3.0).
    
    Related to T3915

Link to build: https://jenkins.softwareheritage.org/job/DLDBZR/job/tests-on-diff/30/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBZR/job/tests-on-diff/30/console

Harbormaster returned this revision to the author for changes because remote builds failed.Feb 9 2022, 2:33 PM
Harbormaster failed remote builds in B26693: Diff 25851!
swh/loader/bzr/loader.py
98

How does this work if clone isn't defined in Breezy 3.0? Is the command line version different from the lib one?

swh/loader/bzr/loader.py
98

oh you mean that breezy underneath uses the bzr stuff?

98

then i guess that won't work.

Hopefully, @Alphare will come up with something neater.