Page MenuHomeSoftware Heritage

Ignore the repository's config
ClosedPublic

Authored by Alphare on Apr 27 2021, 5:30 PM.

Details

Summary

HGRCPATH only tells Mercurial to ignore the user's config files, but
some repositories have a .hg/hgrc file (only in the case that you copy
the files instead of cloning, if present) that is usually used for server-side
configuration. We want to ignore this, since it might affect loading
and ask for hooks that are not there or are otherwise annoying/dangerous,
for example.

Diff Detail

Repository
rDLDHG Mercurial loader
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 21161
Build 32844: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 32843: arc lint + arc unit

Event Timeline

Build has FAILED

Patch application report for D5626 (id=20066)

Could not rebase; Attempt merge onto f03f274065...

Updating f03f274..c6c3b38
Fast-forward
 swh/loader/mercurial/from_disk.py | 9 ++++++++-
 swh/loader/mercurial/hgutil.py    | 3 ++-
 swh/loader/mercurial/utils.py     | 3 ++-
 3 files changed, 12 insertions(+), 3 deletions(-)
Changes applied before test
commit c6c3b386ef246860e9292ab0331aaf32cf72d61b
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Mon Apr 26 23:28:50 2021 +0200

    Ignore the repository's config
    
    `HGRCPATH` only tells Mercurial to ignore the user's config files, but
    some repositories have a `.hg/hgrc` file (only in the case that you copy
    the files instead of cloning, if present) that is usually used for server-side
    configuration. We want to ignore this, since it might affect loading
    and ask for hooks that are not there or are otherwise annoying/dangerous,
    for example.

commit 2ec0206482f46491086791b6b8718d5094cb4d77
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Mon Apr 26 23:26:09 2021 +0200

    Also use minimal env in the new Mercurial loader
    
    The old loader (bundle2 loader) already received this treatment which
    ensures Mercurial doesn't pick up on any user customization, but I
    apparently forgot to apply the same changes to the new one.

commit 1d8b26c042011f3271e451b11b670f37b44e8685
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Tue Apr 27 10:53:56 2021 +0200

    Use billiard instead of stdlib multiprocessing
    
    This circumvents a few celery-related issues, and is consistent with
    what the rest of the codebase does.

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/206/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/206/console

Harbormaster returned this revision to the author for changes because remote builds failed.Apr 27 2021, 5:35 PM
Harbormaster failed remote builds in B21059: Diff 20066!

Build is green

Patch application report for D5626 (id=20079)

Could not rebase; Attempt merge onto f03f274065...

Updating f03f274..250edbb
Fast-forward
 swh/loader/mercurial/from_disk.py         |  9 ++++++++-
 swh/loader/mercurial/hgutil.py            | 18 +++++++++++++-----
 swh/loader/mercurial/tests/test_hgutil.py | 11 +++++++----
 swh/loader/mercurial/utils.py             |  3 ++-
 4 files changed, 30 insertions(+), 11 deletions(-)
Changes applied before test
commit 250edbb11b85a62498dde8def39e84367cd3cebb
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Mon Apr 26 23:28:50 2021 +0200

    Ignore the repository's config
    
    `HGRCPATH` only tells Mercurial to ignore the user's config files, but
    some repositories have a `.hg/hgrc` file (only in the case that you copy
    the files instead of cloning, if present) that is usually used for server-side
    configuration. We want to ignore this, since it might affect loading
    and ask for hooks that are not there or are otherwise annoying/dangerous,
    for example.

commit 457fb88bf36d6c4eedee5e9423f9747e3ea4abf4
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Mon Apr 26 23:26:09 2021 +0200

    Also use minimal env in the new Mercurial loader
    
    The old loader (bundle2 loader) already received this treatment which
    ensures Mercurial doesn't pick up on any user customization, but I
    apparently forgot to apply the same changes to the new one.

commit a89783c52f2e3c44e08018e0c5fa99f54471f994
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Tue Apr 27 10:53:56 2021 +0200

    Use billiard instead of stdlib multiprocessing
    
    This circumvents a few celery-related issues, and is consistent with
    what the rest of the codebase does.

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/211/ for more details.

This revision is now accepted and ready to land.Apr 28 2021, 12:12 PM
This revision was landed with ongoing or failed builds.Apr 30 2021, 11:04 AM
This revision was automatically updated to reflect the committed changes.

Build is green

Patch application report for D5626 (id=20171)

Could not rebase; Attempt merge onto 504ee123ce...

Updating 504ee12..f73d960
Fast-forward
 swh/loader/mercurial/from_disk.py         |  9 ++++++++-
 swh/loader/mercurial/hgutil.py            | 24 ++++++++++++++++++------
 swh/loader/mercurial/tests/test_hgutil.py | 11 +++++++----
 swh/loader/mercurial/utils.py             |  3 ++-
 4 files changed, 35 insertions(+), 12 deletions(-)
Changes applied before test
commit f73d960b0d6c6d24beca8f8a32143cbf54353210
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Mon Apr 26 23:28:50 2021 +0200

    Ignore the repository's config
    
    `HGRCPATH` only tells Mercurial to ignore the user's config files, but
    some repositories have a `.hg/hgrc` file (only in the case that you copy
    the files instead of cloning, if present) that is usually used for server-side
    configuration. We want to ignore this, since it might affect loading
    and ask for hooks that are not there or are otherwise annoying/dangerous,
    for example.

commit aa80a360e236a1a5e155ed83af0bdcc88bf83d74
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Mon Apr 26 23:26:09 2021 +0200

    Also use minimal env in the new Mercurial loader
    
    The old loader (bundle2 loader) already received this treatment which
    ensures Mercurial doesn't pick up on any user customization, but I
    apparently forgot to apply the same changes to the new one.

commit 23260277700970ffabcfaa386e3ce5c619c8294f
Author: Raphaël Gomès <rgomes@octobus.net>
Date:   Tue Apr 27 10:53:56 2021 +0200

    Use billiard instead of stdlib multiprocessing
    
    This circumvents a few celery-related issues, and is consistent with
    what the rest of the codebase does.
    
    stdlib multiprocessing is not able to spawn children from daemonic
    processes, and even says so plainly if you try:
    
    `AssertionError: daemonic processes are not allowed to have children`
    
    This is incompatible with the SWH infrastructure which needs to do this
    exactly. Fortunately, we're already using billiard and celery. I'm
    assuming that there could be other blocking or annoying differences
    between stdlib and billiard, but we will save ourselves the trouble of
    finding out.

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/219/ for more details.

Build is green

Patch application report for D5626 (id=20172)

Rebasing onto 888471483a...

First, rewinding head to replay your work on top of it...
Fast-forwarded diff-target to base-revision-220-D5626.
Changes applied before test

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/220/ for more details.

Build is green

Patch application report for D5626 (id=20173)

Rebasing onto 888471483a...

First, rewinding head to replay your work on top of it...
Fast-forwarded diff-target to base-revision-221-D5626.
Changes applied before test

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/221/ for more details.