HomeSoftware Heritage

utils.split_range: Split into not overlapping ranges

Description

utils.split_range: Split into not overlapping ranges

Existing listers use the is_within_bound [1] method from the base lister.
This method uses inclusive boundaries in all cases.

As some "range" task listers [2] [3] are using split_range function to create
"overlapping" ranges, this can cause concurrent insert issues down the line [4].

This commit adapts the function split_range to make the generated ranges no
longer overlap.

[1]
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/core/lister_base.py$194-199

[2]
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitlab/tasks.py$37-41

[3]
https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitea/tasks.py$36-41

Related to T2577

Details

Provenance
ardumontAuthored on Sep 9 2020, 6:50 PM
ardumontPushed on Sep 10 2020, 11:03 AM
Differential Revision
D3899: utils.split_range: Make computed ranges not overlap
Parents
rDLS725c1fe4ad07: test_utils: Migrate to pytest
Branches
Loading...
Tags
Loading...
Tasks
T2577: Test gitea lister on staging environment
Build Status
Buildable 14984
Build 23086: test-and-buildJenkins console · Jenkins