Differential D8949

svn: Remove use of costly checkout in SvnRepo.export_temporary
ClosedPublic
Actions

Authored by anlambert on Dec 9 2022, 10:59 AM.

Details

Reviewers

ardumont

Group Reviewers

Reviewers

Commits

rDLDSVN5c26c47a4540: svn: Remove use of costly checkout in SvnRepo.export_temporary

Summary

Now that SvnRepo.propget supports URL as target, we can remove the
use of costly checkout operation and directly retrieve the whole
set of svn:externals properties.

This should greatly improve incremental loading of a big repository
in terms of performance.

Depends on D8948

Diff Detail

Repository

rDLDSVN Subversion (SVN) loader

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Dec 9 2022, 10:59 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptDec 9 2022, 10:59 AM

Build is green

Patch application report for D8949 (id=32249)

Could not rebase; Attempt merge onto 91806d4471...

Updating 91806d4..5c26c47
Fast-forward
 swh/loader/svn/svn.py               | 102 ++++++++++++++++++++----------------
 swh/loader/svn/tests/test_loader.py |  53 +++++++++++++++++++
 2 files changed, 109 insertions(+), 46 deletions(-)

Changes applied before test

commit 5c26c47a45404fec2a79c43793c52f26adeea98a
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Dec 8 21:21:51 2022 +0100

    svn: Remove use of costly checkout in SvnRepo.export_temporary
    
    Now that SvnRepo.propget supports URL as target, we can remove the
    use of costly checkout operation and directly retrieve the whole
    set of svn:externals properties.
    
    This should greatly improve incremental loading of a big repository
    in terms of performance.

commit 24d70804b72b62c2531b68bc60db15a75bdb2bd5
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Dec 8 21:18:28 2022 +0100

    svn: Fix propget operation when target is an URL
    
    subvertpy 0.11 has a buggy implementation of propget bindings when
    target is an URL (https://github.com/jelmer/subvertpy/issues/35),
    so as a workaround we implement propget for URL using non buggy
    proplist bindings.

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/369/ for more details.

Harbormaster completed remote builds in B33204: Diff 32249.Dec 9 2022, 11:02 AM

anlambert requested review of this revision.Dec 9 2022, 11:02 AM

ardumont accepted this revision.Dec 12 2022, 11:51 AM

This revision is now accepted and ready to land.Dec 12 2022, 11:51 AM

Closed by commit rDLDSVN5c26c47a4540: svn: Remove use of costly checkout in SvnRepo.export_temporary (authored by anlambert). · Explain WhyDec 12 2022, 2:42 PM

This revision was automatically updated to reflect the committed changes.

anlambert added a commit: rDLDSVN5c26c47a4540: svn: Remove use of costly checkout in SvnRepo.export_temporary.