Page MenuHomeSoftware Heritage

Refactor common loader behavior within swh.model.from_disk.iter_directory function
ClosedPublic

Authored by ardumont on Jul 2 2020, 11:58 AM.

Details

Summary

This refactors the common duplicated code in svn [1] and package loader [2]
within the model. That will decrease maintenance and also fix 1 todo \m/.

[1] D3367

[2] D3368

Depends on D3390

Test Plan

tox

Diff Detail

Repository
rDMOD Data model
Branch
unify-object-type
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13254
Build 20241: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 20240: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D3391 (id=12033)

Could not rebase; Attempt merge onto 40a40f508b...

Updating 40a40f5..35a776a
Fast-forward
 swh/model/from_disk.py            | 48 ++++++++++++++++++++++++++++++++++-----
 swh/model/merkle.py               |  6 ++---
 swh/model/tests/test_from_disk.py | 28 ++++++++++++++++++++++-
 swh/model/tests/test_merkle.py    | 16 ++++++++-----
 4 files changed, 82 insertions(+), 16 deletions(-)
Changes applied before test
commit 35a776a34cc8742d93c0abf8db6771a120083039
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 2 11:57:44 2020 +0200

    Factorize common loader behavior within from_disk.iter_directory

commit 262d84043df0fb0c91082eb94cc91f4d2a8b9902
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 2 10:58:43 2020 +0200

    Unify object_type some more within the merkle and from_disk modules

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/90/ for more details.

Make the returned results a tuple of lists

Build is green

Patch application report for D3391 (id=12034)

Could not rebase; Attempt merge onto 40a40f508b...

Updating 40a40f5..2f1d265
Fast-forward
 swh/model/from_disk.py            | 46 ++++++++++++++++++++++++++++++++++-----
 swh/model/merkle.py               |  6 ++---
 swh/model/tests/test_from_disk.py | 28 +++++++++++++++++++++++-
 swh/model/tests/test_merkle.py    | 16 +++++++++-----
 4 files changed, 80 insertions(+), 16 deletions(-)
Changes applied before test
commit 2f1d2652d2e6651e6498be8aea1bae496a123ebc
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 2 11:57:44 2020 +0200

    Factorize common loader behavior within from_disk.iter_directory

commit 262d84043df0fb0c91082eb94cc91f4d2a8b9902
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 2 10:58:43 2020 +0200

    Unify object_type some more within the merkle and from_disk modules

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/91/ for more details.

ardumont retitled this revision from Factorize common loader behavior within from_disk.iter_directory to Factorize common loader behavior within swh.model.from_disk.iter_directory function.Jul 2 2020, 12:23 PM
ardumont edited the summary of this revision. (Show Details)
anlambert added a subscriber: anlambert.

Looks good, I added some nitpicking comments.

swh/model/from_disk.py
301

You should use model constants here for readability:

if obj_type in (model.Content.object_type, DiskBackedContent.object_type):
306

same here

308

same here

This revision is now accepted and ready to land.Jul 2 2020, 2:44 PM
swh/model/from_disk.py
301

yeah, i'm kinda lazy today ;)

ardumont retitled this revision from Factorize common loader behavior within swh.model.from_disk.iter_directory function to Refactor common loader behavior within swh.model.from_disk.iter_directory function.Jul 2 2020, 2:54 PM
  • Use "refactor" instead of "factorize" term
  • Adapt according to review

Build is green

Patch application report for D3391 (id=12040)

Could not rebase; Attempt merge onto 40a40f508b...

Updating 40a40f5..8863b5c
Fast-forward
 swh/model/from_disk.py            | 46 ++++++++++++++++++++++++++++++++++-----
 swh/model/merkle.py               |  7 +++---
 swh/model/tests/test_from_disk.py | 28 +++++++++++++++++++++++-
 swh/model/tests/test_merkle.py    | 16 +++++++++-----
 4 files changed, 80 insertions(+), 17 deletions(-)
Changes applied before test
commit 8863b5c186ddfdbabfe2b4f9f7d4468819ca474a
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 2 11:57:44 2020 +0200

    Refactor common loader behavior within from_disk.iter_directory

commit 363b1659a6f5fbc32472f0774352d58d068f4282
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 2 10:58:43 2020 +0200

    Unify object_type some more within the merkle and from_disk modules

See https://jenkins.softwareheritage.org/job/DMOD/job/tests-on-diff/93/ for more details.