Differential D5902

Remove origin_get_id method from ProvenanceInterface
ClosedPublic
Actions

Authored by aeviso on Jun 21 2021, 5:15 PM.

Details

Reviewers

douardda

Group Reviewers

Reviewers

Commits

rDPROVd892b29e40c0: Remove origin_get_id method from ProvenanceInterface

Summary

Hide origin.id from the algorithm, thus improvig abstraction from the db layer.

Depends on D5886

Diff Detail

Repository

rDPROV Provenance database

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

aeviso created this revision.Jun 21 2021, 5:15 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJun 21 2021, 5:15 PM

rebase

Build is green

Patch application report for D5902 (id=21175)

Could not rebase; Attempt merge onto 011645221c...

Updating 0116452..fa6977a
Fast-forward
 requirements-swh.txt                               |   2 +-
 swh/provenance/archive.py                          |  42 ++++--
 swh/provenance/graph.py                            | 133 +++++++++++-------
 swh/provenance/model.py                            |  95 ++++++-------
 swh/provenance/origin.py                           |  90 +++++-------
 swh/provenance/postgresql/archive.py               |  70 ++++++----
 swh/provenance/postgresql/provenancedb_base.py     | 153 ++++++++++++---------
 .../postgresql/provenancedb_with_path.py           |  26 ++--
 .../postgresql/provenancedb_without_path.py        |  22 +--
 swh/provenance/provenance.py                       |  73 +++++-----
 swh/provenance/revision.py                         |   8 +-
 swh/provenance/sql/30-schema.sql                   |  10 +-
 swh/provenance/storage/archive.py                  |  72 +++++-----
 swh/provenance/tests/test_isochrone_graph.py       |   4 +-
 14 files changed, 437 insertions(+), 363 deletions(-)

Changes applied before test

commit fa6977a3f5f66162c188668f465385f2a7ba1cc2
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 16:46:51 2021 +0200

    Remove origin_get_id method from ProvenanceInterface
    
    Hide origin.id from the algorithm, thus improvig abstraction from the db layer.

commit 9191002381e097ef8cfd48bb7f7e327963b5f7c3
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 15:51:20 2021 +0200

    Refactor origin-revision layer algorithm to use the new history graph structure

commit 61206fef3c9a6d75967e373ca77973bbe7df4052
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 14:47:30 2021 +0200

    Add history graph structure to be used in the origin-revision layer algorithm

commit 87fbfa18c1884e7ccd732d07304e57494400eaa4
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 14:42:12 2021 +0200

    Fix IsochroneNode hash calculation
    
    There was a bug as the previous implementation was considering mutable attributes.
    This fix allows to replace the list of children by a set instead. Also, all uses
    of `hash_to_hex` were removed from the graph module in favour of bytes' `hex` method.

commit 6a97e7e2e30ec20a7265f1526c3fff8ea70d2cbb
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 15:23:29 2021 +0200

    Remove `hash_to_hex` usage in the revision-content layer
    
    It was only used for debug messaged and it's now replaced by bytes' `hex` method.

commit f2ec1e58c91f1fda1121c6d7c1e29dede6431d45
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jun 16 13:53:25 2021 +0200

    Update backend methods associated to the origin-revision layer

commit fd66d83c119d8f8b283098f6320bf6f68cc7114f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jun 16 11:11:23 2021 +0200

    Fix outdated comments and code styling.

commit 417fd014d96f09b8a20831a6a060cfb408376d5c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 14 14:03:09 2021 +0200

    Refactor RevisionEntry's parents iterator
    
    Make parents a class property and create a separate method to retrieve information
    from the archive, just as it is done for the other model classes

commit f354b65e52ed78b3a637b2632e1b74bb920669be
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 13:51:54 2021 +0200

    Rework ArchiveInterface
    
    Remove Unused methods and fix type annotations. Update Other methods in
    OriginEntry and RevisionEntry accordingly.

commit e6f39d0244b10b49942b0ab93d4628828e343642
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 13:48:24 2021 +0200

    Fix bugs when retrieving parents in RevisionEntry
    
    Convert `Revision.date` from` TimestampWithTimezone` to `datetime` as expected by` RevisionEntry`.
    Create a list with the iterator returned by `ArchiveInterface.revision_get()` before comparison.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/181/ for more details.

Harbormaster completed remote builds in B22162: Diff 21175.Jun 21 2021, 5:27 PM

Build is green

Patch application report for D5902 (id=21178)

Could not rebase; Attempt merge onto 011645221c...

Updating 0116452..e40db60
Fast-forward
 requirements-swh.txt                               |   2 +-
 swh/provenance/archive.py                          |  42 ++++--
 swh/provenance/graph.py                            | 133 +++++++++++-------
 swh/provenance/model.py                            |  95 ++++++-------
 swh/provenance/origin.py                           |  90 +++++-------
 swh/provenance/postgresql/archive.py               |  70 ++++++----
 swh/provenance/postgresql/provenancedb_base.py     | 153 ++++++++++++---------
 .../postgresql/provenancedb_with_path.py           |  26 ++--
 .../postgresql/provenancedb_without_path.py        |  22 +--
 swh/provenance/provenance.py                       |  73 +++++-----
 swh/provenance/revision.py                         |   8 +-
 swh/provenance/sql/30-schema.sql                   |  10 +-
 swh/provenance/storage/archive.py                  |  72 +++++-----
 swh/provenance/tests/test_isochrone_graph.py       |   4 +-
 14 files changed, 437 insertions(+), 363 deletions(-)

Changes applied before test

commit e40db6012a4920dff5ae9ddbd39ca979c59806fa
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 16:46:51 2021 +0200

    Remove origin_get_id method from ProvenanceInterface
    
    Hide origin.id from the algorithm, thus improvig abstraction from the db layer.

commit 532a9cba75fc066b82300a58c3467a44e5cc1809
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 15:51:20 2021 +0200

    Refactor origin-revision layer algorithm to use the new history graph structure

commit 1773a39d2b026d7574ebb69a903652fc36accd07
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 14:47:30 2021 +0200

    Add history graph structure to be used in the origin-revision layer algorithm

commit 87fbfa18c1884e7ccd732d07304e57494400eaa4
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 14:42:12 2021 +0200

    Fix IsochroneNode hash calculation
    
    There was a bug as the previous implementation was considering mutable attributes.
    This fix allows to replace the list of children by a set instead. Also, all uses
    of `hash_to_hex` were removed from the graph module in favour of bytes' `hex` method.

commit 6a97e7e2e30ec20a7265f1526c3fff8ea70d2cbb
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 15:23:29 2021 +0200

    Remove `hash_to_hex` usage in the revision-content layer
    
    It was only used for debug messaged and it's now replaced by bytes' `hex` method.

commit f2ec1e58c91f1fda1121c6d7c1e29dede6431d45
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jun 16 13:53:25 2021 +0200

    Update backend methods associated to the origin-revision layer

commit fd66d83c119d8f8b283098f6320bf6f68cc7114f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jun 16 11:11:23 2021 +0200

    Fix outdated comments and code styling.

commit 417fd014d96f09b8a20831a6a060cfb408376d5c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 14 14:03:09 2021 +0200

    Refactor RevisionEntry's parents iterator
    
    Make parents a class property and create a separate method to retrieve information
    from the archive, just as it is done for the other model classes

commit f354b65e52ed78b3a637b2632e1b74bb920669be
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 13:51:54 2021 +0200

    Rework ArchiveInterface
    
    Remove Unused methods and fix type annotations. Update Other methods in
    OriginEntry and RevisionEntry accordingly.

commit e6f39d0244b10b49942b0ab93d4628828e343642
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 13:48:24 2021 +0200

    Fix bugs when retrieving parents in RevisionEntry
    
    Convert `Revision.date` from` TimestampWithTimezone` to `datetime` as expected by` RevisionEntry`.
    Create a list with the iterator returned by `ArchiveInterface.revision_get()` before comparison.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/183/ for more details.

Harbormaster completed remote builds in B22165: Diff 21178.Jun 21 2021, 5:33 PM

aeviso requested review of this revision.Jun 21 2021, 5:33 PM

rebase

Build is green

Patch application report for D5902 (id=21186)

Could not rebase; Attempt merge onto 011645221c...

Updating 0116452..d892b29
Fast-forward
 requirements-swh.txt                               |   2 +-
 swh/provenance/archive.py                          |  42 ++++--
 swh/provenance/graph.py                            | 133 +++++++++++-------
 swh/provenance/model.py                            |  95 ++++++-------
 swh/provenance/origin.py                           |  90 +++++-------
 swh/provenance/postgresql/archive.py               |  70 ++++++----
 swh/provenance/postgresql/provenancedb_base.py     | 153 ++++++++++++---------
 .../postgresql/provenancedb_with_path.py           |  26 ++--
 .../postgresql/provenancedb_without_path.py        |  22 +--
 swh/provenance/provenance.py                       |  73 +++++-----
 swh/provenance/revision.py                         |   8 +-
 swh/provenance/sql/30-schema.sql                   |  10 +-
 swh/provenance/storage/archive.py                  |  72 +++++-----
 swh/provenance/tests/test_isochrone_graph.py       |   4 +-
 14 files changed, 437 insertions(+), 363 deletions(-)

Changes applied before test

commit d892b29e40c0136aee7a78a2441d0de2dfc91dd6
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 16:46:51 2021 +0200

    Remove origin_get_id method from ProvenanceInterface
    
    Hide origin.id from the algorithm, thus improvig abstraction from the db layer.

commit 86b731bbde4d55cdc6f193d405811dd86cbde555
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 15:51:20 2021 +0200

    Refactor origin-revision layer algorithm to use the new history graph structure

commit f358690963061c8b0afdddd4f67f5dec2b5be53f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 14:47:30 2021 +0200

    Add history graph structure to be used in the origin-revision layer algorithm

commit 78614359c07e61abf9f1d8293644430dcba09273
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 21 14:42:12 2021 +0200

    Fix IsochroneNode hash calculation
    
    There was a bug as the previous implementation was considering mutable attributes.
    This fix allows to replace the list of children by a set instead. Also, all uses
    of `hash_to_hex` were removed from the graph module in favour of bytes' `hex` method.

commit acebbfcf4527e9f69e5a47e7ebb847025078ed2f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 15:23:29 2021 +0200

    Remove `hash_to_hex` usage in the revision-content layer
    
    It was only used for debug messaged and it's now replaced by bytes' `hex` method.

commit a0e6dffc572d46a8b0b4407319eeb5b189cf9dc8
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jun 16 13:53:25 2021 +0200

    Update backend methods associated to the origin-revision layer

commit fd66d83c119d8f8b283098f6320bf6f68cc7114f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jun 16 11:11:23 2021 +0200

    Fix outdated comments and code styling.

commit 417fd014d96f09b8a20831a6a060cfb408376d5c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Jun 14 14:03:09 2021 +0200

    Refactor RevisionEntry's parents iterator
    
    Make parents a class property and create a separate method to retrieve information
    from the archive, just as it is done for the other model classes

commit f354b65e52ed78b3a637b2632e1b74bb920669be
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 13:51:54 2021 +0200

    Rework ArchiveInterface
    
    Remove Unused methods and fix type annotations. Update Other methods in
    OriginEntry and RevisionEntry accordingly.

commit e6f39d0244b10b49942b0ab93d4628828e343642
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jun 17 13:48:24 2021 +0200

    Fix bugs when retrieving parents in RevisionEntry
    
    Convert `Revision.date` from` TimestampWithTimezone` to `datetime` as expected by` RevisionEntry`.
    Create a list with the iterator returned by `ArchiveInterface.revision_get()` before comparison.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/186/ for more details.

Harbormaster completed remote builds in B22173: Diff 21186.Jun 22 2021, 10:51 AM

overall ok but see the comment

swh/provenance/postgresql/provenancedb_base.py
115	I'm a bit surprised that you change the signature of this method but I see nowhere the equivalent modification from callers of this method in this diff. Did I miss something? Or was the type annotation wrong before and this fixes it (in which case the fact mypy did not detect the mistyping before is weird).