Page MenuHomeSoftware Heritage

svn: Optimize propget performance after checkout
ClosedPublic

Authored by anlambert on Feb 3 2022, 4:48 PM.

Details

Summary

The goal here is to recursively get svn:externals property values
from a freshly checked out working copy at a given revision.

But passing a non null revision parameter to proget will make it get
the property values from the remote repository regardless of the checked
out revison and can lead to really long execution times due to a high
amount of network requests.

So force to get the property values from the local working copy
by setting the revision parameter to None.

Related to T611

Diff Detail

Repository
rDLDSVN Subversion (SVN) loader
Branch
propget-performance-fix
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 26507
Build 41453: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 41452: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D7077 (id=25677)

Rebasing onto 87e98c14b3...

Current branch diff-target is up to date.
Changes applied before test
commit 09e83fd94cc03f1e974afc758b40242b3143579d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Feb 3 16:45:28 2022 +0100

    svn: Optimize propget performance after checkout
    
    The goal here is to recursively get svn:externals property values
    from a freshly checked out working copy at a given revision.
    
    But passing a non null revision parameter to proget will make it get
    the property values from the remote repository regardless of the checked
    out revison and can lead to really long execution times due to a high
    amount of network requests.
    
    So force to get the property values from the local working copy
    by setting the revision parameter to None.
    
    Related to T611

See https://jenkins.softwareheritage.org/job/DLDSVN/job/tests-on-diff/274/ for more details.

ardumont added inline comments.
swh/loader/svn/svn.py
238

But to what svn commit is the svn:externals property coming from?

I mean is it consistent with and without the revision?

swh/loader/svn/svn.py
238

yes because we did a checkout of that revision (see line above)

This revision is now accepted and ready to land.Feb 3 2022, 4:56 PM