Page MenuHomeSoftware Heritage
Differential D4643

HgLoaderFromDisk: uneventful load when unchanged
ClosedPublic
Actions

Authored by Alphare on Dec 2 2020, 11:54 AM.

Details

Reviewers
douardda
acezar
ardumont
Group Reviewers
Reviewers
Commits
rDLDHG04a1213f3679: HgLoaderFromDisk: uneventful load when unchanged
Summary

By looking at the previous snapshot heads, loading of an unchanged repository
will be uneventful.

Diff Detail

Repository
rDLDHG Mercurial loader
Branch
arcpatch-D4643
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 19506
Build 30260: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 30259: arc lint + arc unit

Event Timeline

acezar created this revision.Dec 2 2020, 11:54 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptDec 2 2020, 11:54 AM
swh-public-ci added a comment.Dec 2 2020, 11:56 AM

Build is green

Patch application report for D4643 (id=16475)

Rebasing onto 4bf91cff72...

Current branch diff-target is up to date.
Changes applied before test
commit b9af2e3ea95319e2a28eb6c535dd9c9bc291a71a
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Nov 30 14:29:47 2020 +0100

    Make loading an unchanged repository uneventful
    
    By looking at the previous snapshot heads, loading of an unchanged repository
    will be uneventful.

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/132/ for more details.

Harbormaster completed remote builds in B17661: Diff 16475.Dec 2 2020, 11:56 AM
acezar retitled this revision from Make loading an unchanged repository uneventful to HgLoaderFromDisk: uneventful load when unchanged.Dec 2 2020, 3:13 PM
acezar edited the summary of this revision. (Show Details)
acezar added a child revision: D4649: HgLoaderFromDisk: Only load new commits.
douardda requested changes to this revision.Dec 4 2020, 2:26 PM
douardda added a subscriber: douardda.

overall lgtm but see comments

swh/loader/mercurial/from_disk.py
229

should add type annotation

258

nit: why not if not revs: ?

This revision now requires changes to proceed.Dec 4 2020, 2:26 PM
acezar updated this revision to Diff 16565.Dec 4 2020, 4:41 PM
acezar marked 2 inline comments as done.
acezar edited the summary of this revision. (Show Details)

Followup

swh/loader/mercurial/from_disk.py
258

Was not sure __bool__ / __nonzero__ was implemented by the class

swh-public-ci added a comment.Dec 4 2020, 5:02 PM

Build was aborted

Patch application report for D4643 (id=16565)

Rebasing onto 4bf91cff72...

Current branch diff-target is up to date.
Changes applied before test
commit 7af11ca8dcaaec1b65250bfbdb9a9d43d3c588cf
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Mon Nov 30 14:29:47 2020 +0100

    HgLoaderFromDisk: uneventful load when unchanged
    
    By looking at the previous snapshot heads, loading of an unchanged repository
    will be uneventful.

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/134/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/134/console

Harbormaster failed remote builds in B17748: Diff 16565!Dec 4 2020, 5:02 PM
douardda accepted this revision.Dec 4 2020, 5:04 PM
This revision is now accepted and ready to land.Dec 4 2020, 5:04 PM
Alphare updated this revision to Diff 18418.Feb 25 2021, 6:21 PM
Alphare added a subscriber: Alphare.

Fix broken test + rebase (taking over from Antoine's work)

swh-public-ci added a comment.Feb 25 2021, 6:22 PM

Build has FAILED

Patch application report for D4643 (id=18418)

Rebasing onto d8572187e9...

Current branch diff-target is up to date.
Changes applied before test
commit 7a8328ef77cd9ba179e6767602dfc69bfef395ab
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Feb 25 17:35:32 2021 +0100

    HgLoaderFromDisk: uneventful load when unchanged
    
    Summary:
    By looking at the previous snapshot heads, loading of an unchanged repository
    will be uneventful.
    
    Reviewers: #reviewers, douardda
    
    Reviewed By: #reviewers, douardda
    
    Subscribers: douardda
    
    Differential Revision: https://forge.softwareheritage.org/D4643

Link to build: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/168/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/168/console

Harbormaster failed remote builds in B19496: Diff 18418!Feb 25 2021, 6:22 PM
Alphare updated this revision to Diff 18426.Feb 26 2021, 10:43 AM

Fix flake8

swh-public-ci added a comment.Feb 26 2021, 10:45 AM

Build is green

Patch application report for D4643 (id=18426)

Rebasing onto d8572187e9...

Current branch diff-target is up to date.
Changes applied before test
commit d6a7b44fa33ad769d5670f79d7aa53691f43f415
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Feb 25 17:35:32 2021 +0100

    HgLoaderFromDisk: uneventful load when unchanged
    
    Summary:
    By looking at the previous snapshot heads, loading of an unchanged repository
    will be uneventful.
    
    Reviewers: #reviewers, douardda
    
    Reviewed By: #reviewers, douardda
    
    Subscribers: douardda
    
    Differential Revision: https://forge.softwareheritage.org/D4643

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/169/ for more details.

Harbormaster completed remote builds in B19505: Diff 18426.Feb 26 2021, 10:45 AM
Alphare commandeered this revision.Feb 26 2021, 10:45 AM
Alphare added a reviewer: acezar.
ardumont added a subscriber: ardumont.Feb 26 2021, 10:50 AM
ardumont added inline comments.
swh/loader/mercurial/tests/test_from_disk.py
251

swh_config is no longer needed since you inject the swh_storage instance into the loader now.

ardumont requested changes to this revision.Feb 26 2021, 10:53 AM

for the sake of my question, remark ;)

swh/loader/mercurial/from_disk.py
204

can't we make that one for comprehension instead of 2?

self._latest_heads = [
    hash_to_bytes(revision.metadata["node"])
    for revision in self.storage.revision_get(snapshot_branches)
    if revision and revision.metadata
]
This revision now requires changes to proceed.Feb 26 2021, 10:53 AM
ardumont added inline comments.Feb 26 2021, 10:55 AM
swh/loader/mercurial/from_disk.py
195–204

same proposal with the diff format...

(neat trick) the phabricator suggestion edit.

Alphare updated this revision to Diff 18427.Feb 26 2021, 11:07 AM

Improve heads handling + remove useless fixture call

Alphare marked an inline comment as done.Feb 26 2021, 11:08 AM
Alphare added inline comments.
swh/loader/mercurial/from_disk.py
204

Much better, thanks. I saw the patch was accepted and just rebased it, I should have paid more attention. :)

swh-public-ci added a comment.Feb 26 2021, 11:09 AM

Build is green

Patch application report for D4643 (id=18427)

Rebasing onto d8572187e9...

Current branch diff-target is up to date.
Changes applied before test
commit 00b221eaa5aa4f326150ccb3a29259dfafd0eb6f
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Feb 25 17:35:32 2021 +0100

    HgLoaderFromDisk: uneventful load when unchanged
    
    Summary:
    By looking at the previous snapshot heads, loading of an unchanged repository
    will be uneventful.
    
    Reviewers: #reviewers, douardda
    
    Reviewed By: #reviewers, douardda
    
    Subscribers: douardda
    
    Differential Revision: https://forge.softwareheritage.org/D4643

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/170/ for more details.

Harbormaster completed remote builds in B19506: Diff 18427.Feb 26 2021, 11:09 AM
Alphare updated this revision to Diff 18430.Feb 26 2021, 11:20 AM

Rebase

swh-public-ci added a comment.Feb 26 2021, 11:22 AM

Build is green

Patch application report for D4643 (id=18430)

Rebasing onto 1dbb8474ad...

Current branch diff-target is up to date.
Changes applied before test
commit 47c712ecd233611547e64c19783727c6e302bda1
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Feb 25 17:35:32 2021 +0100

    HgLoaderFromDisk: uneventful load when unchanged
    
    Summary:
    By looking at the previous snapshot heads, loading of an unchanged repository
    will be uneventful.
    
    Reviewers: #reviewers, douardda
    
    Reviewed By: #reviewers, douardda
    
    Subscribers: douardda
    
    Differential Revision: https://forge.softwareheritage.org/D4643

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/172/ for more details.

Harbormaster completed remote builds in B19509: Diff 18430.Feb 26 2021, 11:22 AM
ardumont accepted this revision.Feb 26 2021, 12:02 PM

I've added 2 more non-blocking remarks inline (i missed earlier sorry) that you might want to have a look ;)

But otherwise, lgtm.

swh/loader/mercurial/from_disk.py
231
238
This revision is now accepted and ready to land.Feb 26 2021, 12:02 PM
Alphare updated this revision to Diff 18434.Feb 26 2021, 12:07 PM

Add suggested assertion and fix comment typo

swh-public-ci added a comment.Feb 26 2021, 12:09 PM

Build is green

Patch application report for D4643 (id=18434)

Rebasing onto 1dbb8474ad...

Current branch diff-target is up to date.
Changes applied before test
commit 04a1213f3679e7bb1cb3f8cfcc1e7573a60b2419
Author: Antoine Cezar <antoine.cezar@octobus.net>
Date:   Thu Feb 25 17:35:32 2021 +0100

    HgLoaderFromDisk: uneventful load when unchanged
    
    Summary:
    By looking at the previous snapshot heads, loading of an unchanged repository
    will be uneventful.
    
    Reviewers: #reviewers, douardda
    
    Reviewed By: #reviewers, douardda
    
    Subscribers: douardda
    
    Differential Revision: https://forge.softwareheritage.org/D4643

See https://jenkins.softwareheritage.org/job/DLDHG/job/tests-on-diff/174/ for more details.

Harbormaster completed remote builds in B19513: Diff 18434.Feb 26 2021, 12:09 PM
Closed by commit rDLDHG04a1213f3679: HgLoaderFromDisk: uneventful load when unchanged (authored by acezar, committed by Alphare). · Explain WhyFeb 26 2021, 12:18 PM
This revision was automatically updated to reflect the committed changes.
Alphare added a commit: rDLDHG04a1213f3679: HgLoaderFromDisk: uneventful load when unchanged.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
loader/
mercurial/
74 lines90%
Loading...
4 lines97%
Loading...
tests/
33 lines100%
Loading...

Diff 18427

View Options

swh/loader/mercurial/from_disk.py

Loading...
View Options

swh/loader/mercurial/hgutil.py

Loading...
View Options

swh/loader/mercurial/tests/test_from_disk.py

Loading...
About · Developer information · Legal