Page MenuHomeSoftware Heritage
Differential D5973

Reorganize code
ClosedPublic
Actions

Authored by aeviso on Jul 6 2021, 2:51 PM.

Details

Reviewers
olasd
Group Reviewers
Reviewers
Commits
rDPROVcbd38f4a5b51: Improve configuration file format
rDPROV1282b2469501: Reorganize code
rDPROV8c240c7ff2f8: Add missing license string to several files
Summary

Move ProvenanceInterface and ProvenanceStorageInterface to interface.py.
Rename ProvenanceBackend to Provenance and move it to provenance.py.

Depends on D5972

Diff Detail

Event Timeline

aeviso created this revision.Jul 6 2021, 2:51 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJul 6 2021, 2:51 PM
swh-public-ci added a comment.Jul 6 2021, 2:59 PM

Build is green

Patch application report for D5973 (id=21515)

Could not rebase; Attempt merge onto 2e46f7c34b...

Updating 2e46f7c..b37e867
Fast-forward
 swh/provenance/__init__.py                         |  10 +-
 swh/provenance/backend.py                          | 324 ---------------
 swh/provenance/graph.py                            |   2 +-
 swh/provenance/interface.py                        | 313 +++++++++++++++
 swh/provenance/origin.py                           |   2 +-
 swh/provenance/postgresql/provenancedb_base.py     | 133 +++---
 .../postgresql/provenancedb_with_path.py           |   2 +-
 swh/provenance/provenance.py                       | 445 ++++++++++++---------
 swh/provenance/revision.py                         |   2 +-
 swh/provenance/tests/conftest.py                   |   2 +-
 swh/provenance/tests/test_conftest.py              |   2 +-
 swh/provenance/tests/test_history_graph.py         |   2 +-
 swh/provenance/tests/test_isochrone_graph.py       |   2 +-
 swh/provenance/tests/test_provenance_db.py         |   2 +-
 swh/provenance/tests/test_provenance_heuristics.py | 193 ++++-----
 15 files changed, 741 insertions(+), 695 deletions(-)
 delete mode 100644 swh/provenance/backend.py
 create mode 100644 swh/provenance/interface.py
Changes applied before test
commit b37e8673c0d3111cc3a496733cb92c4affdad996
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 14:46:34 2021 +0200

    Reorganize code
    
    Move `ProvenanceInterface` and `ProvenanceStorageInterface` to `interface.py`.
    Rename `ProvenanceBackend` to `Provenance` and move it to `provenance.py`.

commit c246bd709270a6d5d407260f13cc666c18ab9819
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 13:48:50 2021 +0200

    Add `RevisionData` and `RelationData` to improve `ProvenanceStorageInterface`
    
    Get read of `Tuple`s in favour of more declarative objects for passing data.
    Update the usage of the involved methods as well.

commit 3003b15bf120734d8ee51472c594e117a6e9ea28
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 12:30:34 2021 +0200

    Add parametrize `batch` option to heuristics test

commit 9c383756bc5f232a338060f2b6f8872980b71119
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 12:22:21 2021 +0200

    Add testing-specific methods to `ProvenanceStorageInterface`
    
    Refactor `provenance_heuristics` test so that it doesn't perform direct SQL queries,
    hence being able to test different backends in the future.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/251/ for more details.

Harbormaster completed remote builds in B22488: Diff 21515.Jul 6 2021, 2:59 PM
aeviso requested review of this revision.Jul 6 2021, 2:59 PM
aeviso updated this revision to Diff 21579.Jul 9 2021, 1:34 PM

rebase

aeviso added a child revision: D5985: Simplify history graph creation and origin-revision algorithm.Jul 9 2021, 1:36 PM
swh-public-ci added a comment.Jul 9 2021, 1:38 PM

Build is green

Patch application report for D5973 (id=21579)

Could not rebase; Attempt merge onto 2e46f7c34b...

Updating 2e46f7c..bcf19e3
Fast-forward
 swh/provenance/__init__.py                         |  15 +-
 swh/provenance/archive.py                          |   5 +
 swh/provenance/backend.py                          | 324 ---------------
 swh/provenance/cli.py                              |  47 ++-
 swh/provenance/graph.py                            |   7 +-
 swh/provenance/interface.py                        | 318 +++++++++++++++
 swh/provenance/origin.py                           |   7 +-
 swh/provenance/postgresql/archive.py               |   5 +
 swh/provenance/postgresql/provenancedb_base.py     | 138 ++++---
 .../postgresql/provenancedb_with_path.py           |   7 +-
 .../postgresql/provenancedb_without_path.py        |   5 +
 swh/provenance/provenance.py                       | 450 ++++++++++++---------
 swh/provenance/revision.py                         |   7 +-
 swh/provenance/storage/archive.py                  |   5 +
 swh/provenance/tests/conftest.py                   |   2 +-
 swh/provenance/tests/test_conftest.py              |   2 +-
 swh/provenance/tests/test_history_graph.py         |   2 +-
 swh/provenance/tests/test_isochrone_graph.py       |   2 +-
 swh/provenance/tests/test_provenance_db.py         |   2 +-
 swh/provenance/tests/test_provenance_heuristics.py | 193 ++++-----
 20 files changed, 827 insertions(+), 716 deletions(-)
 delete mode 100644 swh/provenance/backend.py
 create mode 100644 swh/provenance/interface.py
Changes applied before test
commit bcf19e3952ad841ed4bb4d78504ea70b1b4fb443
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jul 8 17:29:12 2021 +0200

    Improve configuration file format

commit 8563c9e2915fd785d831e27ca12ed1c896fd4c2e
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jul 8 17:24:39 2021 +0200

    Add missing license string to several files

commit b37e8673c0d3111cc3a496733cb92c4affdad996
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 14:46:34 2021 +0200

    Reorganize code
    
    Move `ProvenanceInterface` and `ProvenanceStorageInterface` to `interface.py`.
    Rename `ProvenanceBackend` to `Provenance` and move it to `provenance.py`.

commit c246bd709270a6d5d407260f13cc666c18ab9819
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 13:48:50 2021 +0200

    Add `RevisionData` and `RelationData` to improve `ProvenanceStorageInterface`
    
    Get read of `Tuple`s in favour of more declarative objects for passing data.
    Update the usage of the involved methods as well.

commit 3003b15bf120734d8ee51472c594e117a6e9ea28
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 12:30:34 2021 +0200

    Add parametrize `batch` option to heuristics test

commit 9c383756bc5f232a338060f2b6f8872980b71119
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 12:22:21 2021 +0200

    Add testing-specific methods to `ProvenanceStorageInterface`
    
    Refactor `provenance_heuristics` test so that it doesn't perform direct SQL queries,
    hence being able to test different backends in the future.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/252/ for more details.

Harbormaster completed remote builds in B22554: Diff 21579.Jul 9 2021, 1:38 PM
aeviso updated this revision to Diff 21583.Jul 9 2021, 2:43 PM

rebase

swh-public-ci added a comment.Jul 9 2021, 2:50 PM

Build is green

Patch application report for D5973 (id=21583)

Could not rebase; Attempt merge onto 2e46f7c34b...

Updating 2e46f7c..cbd38f4
Fast-forward
 swh/provenance/__init__.py                         |  15 +-
 swh/provenance/archive.py                          |   5 +
 swh/provenance/backend.py                          | 324 ---------------
 swh/provenance/cli.py                              |  47 ++-
 swh/provenance/graph.py                            |   7 +-
 swh/provenance/interface.py                        | 303 ++++++++++++++
 swh/provenance/origin.py                           |   7 +-
 swh/provenance/postgresql/archive.py               |   5 +
 swh/provenance/postgresql/provenancedb_base.py     | 138 ++++---
 .../postgresql/provenancedb_with_path.py           |   7 +-
 .../postgresql/provenancedb_without_path.py        |   5 +
 swh/provenance/provenance.py                       | 450 ++++++++++++---------
 swh/provenance/revision.py                         |   7 +-
 swh/provenance/storage/archive.py                  |   5 +
 swh/provenance/tests/conftest.py                   |   2 +-
 swh/provenance/tests/test_conftest.py              |   2 +-
 swh/provenance/tests/test_history_graph.py         |   2 +-
 swh/provenance/tests/test_isochrone_graph.py       |   2 +-
 swh/provenance/tests/test_provenance_db.py         |   2 +-
 swh/provenance/tests/test_provenance_heuristics.py | 193 ++++-----
 20 files changed, 812 insertions(+), 716 deletions(-)
 delete mode 100644 swh/provenance/backend.py
 create mode 100644 swh/provenance/interface.py
Changes applied before test
commit cbd38f4a5b51ede687b4ad713df3d26d2bded0b2
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jul 8 17:29:12 2021 +0200

    Improve configuration file format

commit 8c240c7ff2f8003fc01c1ffe18d5badf282f2de0
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Thu Jul 8 17:24:39 2021 +0200

    Add missing license string to several files

commit 1282b24695016702a59b77075ecc9c27e6f5063f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 14:46:34 2021 +0200

    Reorganize code
    
    Move `ProvenanceInterface` and `ProvenanceStorageInterface` to `interface.py`.
    Rename `ProvenanceBackend` to `Provenance` and move it to `provenance.py`.

commit c96beb57884d19c7fd1fa74aa366f472187419ff
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 13:48:50 2021 +0200

    Add `RevisionData` and `RelationData` to improve `ProvenanceStorageInterface`
    
    Get read of `Tuple`s in favour of more declarative objects for passing data, and
    update the usage of the involved methods.
    
    Turn `ProvenanceResult` into a dataclass as well.

commit 3003b15bf120734d8ee51472c594e117a6e9ea28
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 12:30:34 2021 +0200

    Add parametrize `batch` option to heuristics test

commit 9c383756bc5f232a338060f2b6f8872980b71119
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 6 12:22:21 2021 +0200

    Add testing-specific methods to `ProvenanceStorageInterface`
    
    Refactor `provenance_heuristics` test so that it doesn't perform direct SQL queries,
    hence being able to test different backends in the future.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/256/ for more details.

Harbormaster completed remote builds in B22558: Diff 21583.Jul 9 2021, 2:50 PM
olasd accepted this revision.Jul 12 2021, 9:35 AM
This revision is now accepted and ready to land.Jul 12 2021, 9:35 AM
aeviso added a child revision: D5986: Add provenance storage server/client classes.Jul 12 2021, 11:06 AM
aeviso removed a child revision: D5985: Simplify history graph creation and origin-revision algorithm.Jul 12 2021, 11:10 AM
Closed by commit rDPROV8c240c7ff2f8: Add missing license string to several files (authored by aeviso). · Explain WhyJul 12 2021, 11:19 AM
This revision was automatically updated to reflect the committed changes.
aeviso added a commit: rDPROV8c240c7ff2f8: Add missing license string to several files.
aeviso added a commit: rDPROV1282b2469501: Reorganize code.
aeviso added a commit: rDPROVcbd38f4a5b51: Improve configuration file format.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
provenance/
15 lines58%
Loading...
5 lines--
--
47 lines46%
Loading...
7 lines100%
Loading...
interface.py
provenance.py
5 lines66%
Loading...
7 lines97%
Loading...
postgresql/
5 lines100%
Loading...
7 lines82%
Loading...
7 lines100%
Loading...
5 lines100%
Loading...
476 lines92%
Loading...
7 lines94%
Loading...
storage/
5 lines97%
Loading...
tests/
2 lines97%
Loading...
2 lines100%
Loading...
2 lines100%
Loading...
2 lines100%
Loading...
2 lines100%
Loading...
2 lines100%
Loading...

Diff 21583

View Options

swh/provenance/__init__.py

Loading...
View Options

swh/provenance/archive.py

Loading...
View Options

swh/provenance/backend.py

Loading...
View Options

swh/provenance/cli.py

Loading...
View Options

swh/provenance/graph.py

Loading...
View Options

swh/provenance/interface.py

Loading...
View Options

swh/provenance/origin.py

Loading...
View Options

swh/provenance/postgresql/archive.py

Loading...
View Options

swh/provenance/postgresql/provenancedb_base.py

Loading...
View Options

swh/provenance/postgresql/provenancedb_with_path.py

Loading...
View Options

swh/provenance/postgresql/provenancedb_without_path.py

Loading...
View Options

swh/provenance/provenance.py

Loading...
View Options

swh/provenance/revision.py

Loading...
View Options

swh/provenance/storage/archive.py

Loading...
View Options

swh/provenance/tests/conftest.py

Loading...
View Options

swh/provenance/tests/test_conftest.py

Loading...
View Options

swh/provenance/tests/test_history_graph.py

Loading...
View Options

swh/provenance/tests/test_isochrone_graph.py

Loading...
View Options

swh/provenance/tests/test_provenance_db.py

Loading...
View Options

swh/provenance/tests/test_provenance_heuristics.py

Loading...
About · Developer information · Legal