Page MenuHomeSoftware Heritage

Rename PostgreSQL backend and code styling
ClosedPublic

Authored by aeviso on Aug 13 2021, 11:07 AM.

Details

Summary

Fix bugs in ProvenanceStoragePostgreSql class

location_get was decoding paths to string instead of returning bytes directly as the interface requires.
revision_get was exposing internal origins id instead of joining with the origin table to retrive their sha1s.

Fix module requirements

types-Werkzeug is only required for testing.

Depends on D6071.

Diff Detail

Event Timeline

Build is green

Patch application report for D6084 (id=22032)

Could not rebase; Attempt merge onto 3431de8f49...

Updating 3431de8..5ea920a
Fast-forward
 docs/index.rst                                     |   2 +-
 requirements-test.txt                              |   1 +
 requirements.txt                                   |   1 -
 swh/provenance/__init__.py                         |   6 +-
 swh/provenance/api/server.py                       |   4 +-
 swh/provenance/cli.py                              |   2 +-
 swh/provenance/graph.py                            | 112 ++++---
 swh/provenance/origin.py                           |  22 +-
 .../postgresql/{provenancedb.py => provenance.py}  |  23 +-
 swh/provenance/sql/40-funcs.sql                    | 266 +++++++--------
 swh/provenance/tests/conftest.py                   |   8 +-
 .../data/history_graphs_with-merges_visits-01.yaml | 371 ++++++++++++---------
 swh/provenance/tests/test_conftest.py              |   3 +-
 swh/provenance/tests/test_history_graph.py         |  31 +-
 swh/provenance/tests/test_provenance_db.py         |   4 +-
 .../tests/test_revision_content_layer.py           |   6 +-
 16 files changed, 456 insertions(+), 406 deletions(-)
 rename swh/provenance/postgresql/{provenancedb.py => provenance.py} (94%)
Changes applied before test
commit 5ea920ac03a141a956ab6f2c2ed1845281d8a368
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Aug 13 10:45:19 2021 +0200

    Fix module requirements
    
    `types-Werkzeug` is only required for testing.

commit b4baa4a5dfb535936b7450fed7873003dda9c6ad
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Aug 13 10:35:11 2021 +0200

    Fix bugs in ProvenanceStoragePostgreSql class
    
    `location_get` was decoding paths to string instead of returning bytes directly as the interface requires.
    `revision_get` was exposing internal origins id instead of joining with the `origin` table to retrive their sha1s.

commit 73bd38eeb8041044f9934e74b46579700da4aa2c
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 10 17:06:11 2021 +0200

    Rename PostgreSQL backend and code styling

commit b0951e2e2199afa6bc3bd72a7c7f2c2e6bb58cb7
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Mon Aug 9 15:32:10 2021 +0200

    Revisit history graph implementation
    
    A new class was added to properly represent a history graph (a tree of
    `HistoryNode`s was used until now), and the origin-revision layer algorithm
    was revisited to avoid processing with a merged path in the graph.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/294/ for more details.

douardda added a subscriber: douardda.

Please don't mix fixes with codestyling/renaming revisions in a single diff, it makes the review much harder.

For the fix of revision_get, there should be a test.

swh/provenance/__init__.py
74

please keep support for the former name with a deprecation warning

This revision now requires changes to proceed.Aug 13 2021, 11:35 AM
aeviso added a subscriber: jayeshv.

For the fix of revision_get, there should be a test.

The test is coming later from @jayeshv mongodb branch. This is needed precisely for that development to work but it also concerns the ongoing refactoring of postgresql backend.

I will appreciate not being unnecessary blocked.

For the fix of revision_get, there should be a test.

The test is coming later from @jayeshv mongodb branch.

that is no excuse for not adding a test in this revision.

This is needed precisely for that development to work

There is no way I, as a reviewer, could have guessed it.

but it also concerns the ongoing refactoring of postgresql backend.

same

I will appreciate not being unnecessary blocked.

really?

Add deprecated warning for "local" storage class

Build is green

Patch application report for D6084 (id=22053)

Rebasing onto b0951e2e21...

Current branch diff-target is up to date.
Changes applied before test
commit 2b62b1635651aa76211ce2d81b7d4e5ba0da63e6
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Aug 13 10:45:19 2021 +0200

    Fix module requirements
    
    `types-Werkzeug` is only required for testing.

commit 68e1c8ffddf6264923b28bef9f6836c01a5d8a12
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Aug 13 10:35:11 2021 +0200

    Fix bugs in ProvenanceStoragePostgreSql class
    
    `location_get` was decoding paths to string instead of returning bytes directly as the interface requires.
    `revision_get` was exposing internal origins id instead of joining with the `origin` table to retrive their sha1s.

commit 2086effea8427c256e472035241b1d8d7e121bd9
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 10 17:06:11 2021 +0200

    Rename PostgreSQL backend and code styling

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/296/ for more details.

This revision is now accepted and ready to land.Aug 18 2021, 10:23 AM