Page MenuHomeSoftware Heritage

Remove overhead of to_dict/from_dict in test_snapshot_large.
ClosedPublic

Authored by vlorentz on Jul 9 2020, 6:02 PM.

Details

Summary

This should make it fast enough not to exceed the deadline.

Depends on D3484. (To actually be under the deadline.)

Diff Detail

Repository
rDSTO Storage manager
Branch
snapshot-perf
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13544
Build 20730: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 20729: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D3485 (id=12323)

Could not rebase; Attempt merge onto c3803ef8f7...

Updating c3803ef8..de38cd11
Fast-forward
 swh/storage/in_memory.py                 | 27 ++++++++++++++-------------
 swh/storage/tests/algos/test_snapshot.py | 17 ++++++++---------
 2 files changed, 22 insertions(+), 22 deletions(-)
Changes applied before test
commit de38cd112611e027909b982a6b46ae2d2e3897ec
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 9 18:02:10 2020 +0200

    Remove overhead of to_dict/from_dict in test_snapshot_large.
    
    This should make it fast enough not to exceed the deadline.

commit e415488900bde00db6fe519c76616feca8accd2d
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Thu Jul 9 17:58:12 2020 +0200

    in_memory: Fix quadratic run time in snapshot_get_branches.
    
    snapshot.branches is now an ImmutableDict, which is backed by
    a tuple of tuples; so random accesses now take a linear time
    instead of a constant time.
    
    This commit replaces random accesses with a single scan of all
    the items, and does existence checks in a set instead.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/437/ for more details.

This revision is now accepted and ready to land.Jul 9 2020, 6:14 PM