mercurial.from_disk: Filter out globally already seen hg_nodeid
AbandonedPublicDraft
Actions

Authored by ardumont on Sep 6 2021, 1:28 PM.

Details

Reviewers

Group Reviewers

Summary

This exploits the extid mappings for all seen origins and does not limit the filtering
to only happen when we visit an already seen origin.

In another commit, this also drops an unnecessary flush instruction.

Test Plan

tox (failing as now plenty of uneventful events happen instead of eventful,
probably either some missing reset or some scenario change needs to happen)

Diff Detail

Repository

rDLDHG Mercurial loader

Branch

master

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 23369
Build 36466: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 36465: arc lint + arc unit

Unit TestsFailed
View All

	Time	Test
	101 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_closed_branch_incremental swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cff6ad0f0> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_closed_branch_incremental0')
	107 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[example] swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cff988fd0> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_example_0')
	1,559 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[hello] swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cffb7f588> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_hello_0')
	98 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[the-sandbox] swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cffaeab38> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_the_sandbox_0')
	107 ms	Jenkins > .tox.py3.lib.python3.7.site-packages.swh.loader.mercurial.tests.test_from_disk::test_examples[transplant] swh_storage = <swh.storage.proxies.filter.FilteringProxyStorage object at 0x7f4cffa7b5f8> datadir = '/var/lib/jenkins/workspace/DLDHG/tests-on-diff/.tox/py3/lib/python3.7/site-packages/swh/loader/mercurial/tests/data' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_examples_transplant_0')
		View Full Test Results (17 Failed · 39 Passed)

Event Timeline

ardumont created this revision.Sep 6 2021, 1:28 PM

Build has FAILED

Patch application report for D6188 (id=22415)

Rebasing onto 9be124af21...

Current branch diff-target is up to date.

Changes applied before test

commit f4a524465aa27e36cc1fa0838f59b9b1857b1038
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Sep 6 13:23:52 2021 +0200

    mercurial.from_disk: Filter out already seen hg_nodeid
    
    This exploits the extid mappings for all seen origins and does not limit the filtering
    to only happen when we visit an already seen origin.

commit c678bd573b8f5106ea3440b15f2b3c26f1e65a7a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Mon Sep 6 13:19:29 2021 +0200

    mercurial.from_disk: Drop unnecessary flush instruction

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/279/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/279/console

Harbormaster returned this revision to the author for changes because remote builds failed.Sep 6 2021, 1:30 PM

Harbormaster failed remote builds in B23369: Diff 22415!

ardumont abandoned this revision.Sep 6 2021, 1:49 PM