Differential D256

mercurial bundle20 parser/loader
AbandonedPublic
Actions

Authored by ardumont on Oct 16 2017, 9:58 PM.

Details

Reviewers

fiendish

Group Reviewers

Reviewers

Maniphest Tasks

T329: hg / mercurial loader

Commits

rDLDHG6b5527e45966: fast loader

Summary

A new mercurial loader. It works directly on version 2 bundle files, which is (I believe) the current mercurial transport format. This is much faster than bouncing through hglib for everything. There are a few bits and variables to quantify. See especially any TODO and NOTE comments, of course.

No rush on the review. It's a lot of code all at once. ^o^

After review, will need to switch DEBUG in bundle20_loader.py from True to False.

No official unit tests yet, but there is validation code (bundle20_loader_verifier.py) for contents and directories comparing against hglib output.

Diff Detail

Lint

Lint Skipped

Unit

Unit Tests Skipped

Event Timeline

fiendish created this revision.Oct 16 2017, 9:58 PM

fiendish created this object with edit policy "Developers (Project)".

fiendish edited the summary of this revision. (Show Details)Oct 16 2017, 10:01 PM

fiendish edited reviewers, added: Reviewers; removed: Mercurial loader, Core Loader.

fiendish edited the summary of this revision. (Show Details)

fiendish added a commit: rDLDHG6b5527e45966: fast loader.Oct 16 2017, 10:04 PM

fiendish added a task: T329: hg / mercurial loader.Oct 16 2017, 10:06 PM

go back to creation of chunked_reader

fiendish edited the summary of this revision. (Show Details)Oct 16 2017, 10:28 PM

I don't have any experience with mercurial so I can't comment the code..
I noticed that a new type of revision is created in the process so we should
add a revision type 'hg' to the storage enums

I don't have any experience with mercurial so I can't comment the code..
I noticed that a new type of revision is created in the process so we should
add a revision type 'hg' to the storage enums

Thanks.
It's done btw (rDSTOafcbe940edde5bc6153d07bf748125ac47d161f8).

Already merged in master.

Nice work ,thanks.

I've made a run to adapt, fixed storage call errors, debian packaged it and deployed.
The git log will be helpful to check those adaptations

The slow_reader needs still some work because, as avi mentioned, it's so slow.
It's probably currently broken (e.g. some functions still used in it were deleted).
I've fixed some but did not make a full pass. Not sure, it's worth fixing it either.

I mostly concentrated on the bundle loader.