Page MenuHomeSoftware Heritage

Normalize _add() methods of the ProvenanceStorage interface
ClosedPublic

Authored by douardda on Oct 11 2022, 12:24 PM.

Details

Summary

make them all accept a Dict[Sha1Git, xxx] as argument, ie:

  • remove support for Iterable[bytes] in revision_add, and
  • replace Iterable[bytes] by Dict[Sha1Git, bytes] for location_add

Currently, the sha1 of location path in location_add() is not really
used by any backend, so the computation of said hashed is a waste of
resource, but it makes the API of this interface much more consistent
which will be helpful for coming features (like kafka journal).

Related to T4616

Diff Detail

Event Timeline

Build has FAILED

Patch application report for D8656 (id=31244)

Rebasing onto 6f4a193e90...

Current branch diff-target is up to date.
Changes applied before test
commit d182fcdbc532f1769bec129b2e37c36a59d021de
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 7 14:51:09 2022 +0200

    Normalize _add() methods of the ProvenanceStorage interface
    
    make them all accept a Dict[Sha1Git, xxx] as argument, ie:
    
    - remove support for Iterable[bytes] in revision_add, and
    - replace Iterable[bytes] by Dict[Sha1Git, bytes] for location_add
    
    Currently, the sha1 of location path in location_add() is not really
    used by any backend, so the computation of said hashed is a waste of
    resource, but it makes the API of this interface much more consistent
    which will be helpful for coming features (like kafka journal).

Link to build: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/673/
See console output for more information: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/673/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 11 2022, 12:31 PM
Harbormaster failed remote builds in B32196: Diff 31244!

Build has FAILED

Patch application report for D8656 (id=31249)

Rebasing onto 6f4a193e90...

Current branch diff-target is up to date.
Changes applied before test
commit 6b539ecbe1673caf539e0636de76881a3f8ed171
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 7 14:51:09 2022 +0200

    Normalize _add() methods of the ProvenanceStorage interface
    
    make them all accept a Dict[Sha1Git, xxx] as argument, ie:
    
    - remove support for Iterable[bytes] in revision_add, and
    - replace Iterable[bytes] by Dict[Sha1Git, bytes] for location_add
    
    Currently, the sha1 of location path in location_add() is not really
    used by any backend, so the computation of said hashed is a waste of
    resource, but it makes the API of this interface much more consistent
    which will be helpful for coming features (like kafka journal).

Link to build: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/676/
See console output for more information: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/676/console

Harbormaster returned this revision to the author for changes because remote builds failed.Oct 11 2022, 2:07 PM
Harbormaster failed remote builds in B32201: Diff 31249!

Build is green

Patch application report for D8656 (id=31249)

Rebasing onto 6f4a193e90...

Current branch diff-target is up to date.
Changes applied before test
commit 6b539ecbe1673caf539e0636de76881a3f8ed171
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 7 14:51:09 2022 +0200

    Normalize _add() methods of the ProvenanceStorage interface
    
    make them all accept a Dict[Sha1Git, xxx] as argument, ie:
    
    - remove support for Iterable[bytes] in revision_add, and
    - replace Iterable[bytes] by Dict[Sha1Git, bytes] for location_add
    
    Currently, the sha1 of location path in location_add() is not really
    used by any backend, so the computation of said hashed is a waste of
    resource, but it makes the API of this interface much more consistent
    which will be helpful for coming features (like kafka journal).

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/679/ for more details.

vlorentz added a subscriber: vlorentz.
vlorentz added inline comments.
swh/provenance/provenance.py
6

I'm not a huge fan of import ... from with stdlib modules in general, but here in particular there is the extra issue of sha1 being a common variable name

(and then you can undo the variable name change below)

swh/provenance/tests/test_provenance_storage.py
6

ditto

This revision is now accepted and ready to land.Oct 11 2022, 3:14 PM
swh/provenance/provenance.py
6

yeah I guess you are right, let me change this (thx)

Build is green

Patch application report for D8656 (id=31270)

Rebasing onto 6f4a193e90...

Current branch diff-target is up to date.
Changes applied before test
commit 2bd74fc7d97d40d7132a6530cd0078e3ffb8c614
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 7 14:51:09 2022 +0200

    Normalize _add() methods of the ProvenanceStorage interface
    
    make them all accept a Dict[Sha1Git, xxx] as argument, ie:
    
    - remove support for Iterable[bytes] in revision_add, and
    - replace Iterable[bytes] by Dict[Sha1Git, bytes] for location_add
    
    Currently, the sha1 of location path in location_add() is not really
    used by any backend, so the computation of said hashed is a waste of
    resource, but it makes the API of this interface much more consistent
    which will be helpful for coming features (like kafka journal).

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/681/ for more details.

also rename ProvenanceInterface.directory_xxx_flattenned as directory_xxx_flattened

(included in this Diff by lazyness)

Build is green

Patch application report for D8656 (id=31273)

Rebasing onto 6f4a193e90...

Current branch diff-target is up to date.
Changes applied before test
commit 7e6a62c990b76ac63ee53be1f4c1c147bba4b806
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Oct 11 16:30:46 2022 +0200

    Rename ProvenanceInterface.directory_xxx_flattenned as directory_xxx_flattened
    
    and fix all occurrences of the typo.

commit 2bd74fc7d97d40d7132a6530cd0078e3ffb8c614
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Oct 7 14:51:09 2022 +0200

    Normalize _add() methods of the ProvenanceStorage interface
    
    make them all accept a Dict[Sha1Git, xxx] as argument, ie:
    
    - remove support for Iterable[bytes] in revision_add, and
    - replace Iterable[bytes] by Dict[Sha1Git, bytes] for location_add
    
    Currently, the sha1 of location path in location_add() is not really
    used by any backend, so the computation of said hashed is a waste of
    resource, but it makes the API of this interface much more consistent
    which will be helpful for coming features (like kafka journal).

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/683/ for more details.