Page MenuHomeSoftware Heritage

mercurial.from_disk: Filter out globally already seen hg_nodeid
AbandonedPublicDraft

Authored by ardumont on Sep 6 2021, 1:28 PM.

Details

Reviewers
olasd
Group Reviewers
Reviewers
Summary

This exploits the extid mappings for all seen origins and does not limit the filtering
to only happen when we visit an already seen origin.

In another commit, this also drops an unnecessary flush instruction.

Test Plan

tox (failing as now plenty of uneventful events happen instead of eventful,
probably either some missing reset or some scenario change needs to happen)

Diff Detail

Repository
rDLDHG Mercurial loader
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 23369
Build 36466: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 36465: arc lint + arc unit

Unit TestsFailed

TimeTest
101 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_closed_branch_incremental
swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cff6ad0f0> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_closed_branch_incremental0')
107 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[example]
swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cff988fd0> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_example_0')
1,559 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[hello]
swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cffb7f588> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_hello_0')
98 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[the-sandbox]
swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cffaeab38> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_the_sandbox_0')
107 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[transplant]
swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cffa7b5f8> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_transplant_0')
View Full Test Results (17 Failed · 39 Passed)

Event Timeline

Build has FAILED

Patch application report for D6188 (id=22415)

Rebasing onto 9be124af21...

Current branch diff-target is up to date.
Changes applied before test
commit f4a524465aa27e36cc1fa0838f59b9b1857b1038
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Sep 6 13:23:52 2021 +0200

    mercurial.from_disk: Filter out already seen hg_nodeid
    
    This exploits the extid mappings for all seen origins and does not limit the filtering
    to only happen when we visit an already seen origin.

commit c678bd573b8f5106ea3440b15f2b3c26f1e65a7a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Sep 6 13:19:29 2021 +0200

    mercurial.from_disk: Drop unnecessary flush instruction

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/279/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/279/console

Harbormaster returned this revision to the author for changes because remote builds failed.Sep 6 2021, 1:30 PM
Harbormaster failed remote builds in B23369: Diff 22415!