Page MenuHomeSoftware Heritage
Differential D6054

Add test for the different `ProvenanceStorageInterface` implementations
ClosedPublic
Actions

Authored by aeviso on Aug 3 2021, 4:45 PM.

Details

Reviewers
douardda
Group Reviewers
Reviewers
Commits
rDPROV3431de8f4931: Add test for the different `ProvenanceStorageInterface` implementations
rDPROV0e10aa4c7098: Fix bug on `ProvenanceDB._relation_get` methods when `reverse=True`
rDPROVac6439aa7de2: Make `provenance` fixture use a fixed storage backend for testing
Summary

Make provenance fixture use a fixed storage backend for testing

The parametrized storage fixture will be used to test all possible storage
implementations against the single fixed one in provenance.storage.

Add test for the different ProvenanceStorageInterface implementations

All possible implementations are instantiated in the storage fixture and tested
against the reference on in provenance. storage fixture. The latter is in turn
tested against the actual provenance algorithms.

Depends on D6053.

Diff Detail

Event Timeline

aeviso created this revision.Aug 3 2021, 4:45 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptAug 3 2021, 4:46 PM
swh-public-ci added a comment.Aug 3 2021, 4:56 PM

Build is green

Patch application report for D6054 (id=21906)

Could not rebase; Attempt merge onto 509280132c...

Updating 5092801..d1a70ec
Fast-forward
 mypy.ini                                           |   3 +
 requirements.txt                                   |   1 +
 setup.py                                           |   3 +-
 swh/provenance/api/serializers.py                  |  11 +-
 swh/provenance/api/server.py                       |  35 ++-
 swh/provenance/archive.py                          |   3 +
 swh/provenance/cli.py                              |   2 -
 swh/provenance/postgresql/archive.py               |   8 +-
 swh/provenance/postgresql/provenancedb_base.py     |   8 +-
 .../postgresql/provenancedb_with_path.py           |   8 +-
 .../postgresql/provenancedb_without_path.py        |  14 +-
 swh/provenance/storage/archive.py                  |   2 +-
 swh/provenance/tests/conftest.py                   | 223 +++------------
 swh/provenance/tests/data/README.md                |   2 +-
 swh/provenance/tests/data/generate_repo.py         |   9 +-
 .../tests/data/generate_storage_from_git.py        |  34 ++-
 .../data/origin-revision_with-merges_visits-01.txt |  67 +++++
 swh/provenance/tests/data/with-merges.msgpack      | Bin 7501 -> 10327 bytes
 swh/provenance/tests/test_archive_interface.py     |   4 +-
 swh/provenance/tests/test_cli.py                   |   5 +-
 swh/provenance/tests/test_conftest.py              |  18 +-
 swh/provenance/tests/test_history_graph.py         |   4 +-
 swh/provenance/tests/test_isochrone_graph.py       |   4 +-
 swh/provenance/tests/test_origin_iterator.py       |  36 +--
 swh/provenance/tests/test_origin_revision_layer.py | 190 +++++++++++++
 swh/provenance/tests/test_provenance_db.py         |  19 --
 swh/provenance/tests/test_provenance_storage.py    | 302 ++++++++++++++++++++-
 ...euristics.py => test_revision_content_layer.py} | 154 +++++++++--
 swh/provenance/tests/test_revision_iterator.py     |   6 +-
 29 files changed, 854 insertions(+), 321 deletions(-)
 create mode 100644 swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt
 create mode 100644 swh/provenance/tests/test_origin_revision_layer.py
 rename swh/provenance/tests/{test_provenance_heuristics.py => test_revision_content_layer.py} (70%)
Changes applied before test
commit d1a70eca45d632364c10b0ad4e5e79561a18926f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:37:39 2021 +0200

    Add test for the different `ProvenanceStorageInterface` implementations
    
    All possible implementations are instantiated in the `storage` fixture and tested
    against the reference on in `provenance. storage` fixture. The latter is in turn
    tested against the actual provenance algorithms.

commit f394b7a7396a86bd908044cd1c1755fac46d26e8
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:31:12 2021 +0200

    Make `provenance` fixture use a fixed storage backend for testing
    
    The parametrized `storage` fixture will be used to test all possible storage
    implementations against the single fixed one in `provenance.storage`.

commit 70651f45092ae590ee8a66febf4543cb37a0627f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 15:37:43 2021 +0200

    Refactor the use of archive `Storage` object for testing
    
    Remove any use of `swh.model.tests.swh_model_data.TEST_OBJECTS`, and keep a single fixture for
    `ArchiveInterface` objects based on `ArchiveStorage` (using `swh_storage` fixture as parameter).
    This avoids duplicating every test for each `ArchiveInterface` implementation. All
    implementations of `ArchiveInterface` are tested against each other in `test_archive_interface`.
    
    Also, `ArchiveStorage` now has a `storage` attribute to access the underlying `StorageInterface`
    object, so that test no longer needs to receive `archive` and `swh_storage` separately. Each
    test is responsible for filling `archive. storage` with the required data specific to the test.

commit b738c4610cb84f3f5d10a16f667e4522f7d49377
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit 16d91e21ff1429f897f953e122f6d21e2def0eb5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 92237ff7db1d74aa42c62b5bff4575f2ee0f4087
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/282/ for more details.

Harbormaster completed remote builds in B22872: Diff 21906.Aug 3 2021, 4:56 PM
aeviso requested review of this revision.Aug 3 2021, 4:56 PM
douardda requested changes to this revision.Aug 5 2021, 12:27 PM
douardda added a subscriber: douardda.

overall ok, but I'd like to see the comments about fixtures addressed first.

Thx

swh/provenance/tests/conftest.py
63

since there is no namespace for pytest fixtures, I'd rather keep a "provenance" in the name of this later. storage is way too generic.

85

I don't understand the need for this postgresql2 fixture (which name is not great either).

Why create a new postgresql proc instead of using the (default) main one (postgresql_proc)?

something like:

from pytest_postgresql.factories import postgresql

[...]

provenance_postgresql = postgresql(dbname="provenance")

should be enough I think.

This revision now requires changes to proceed.Aug 5 2021, 12:27 PM
aeviso updated this revision to Diff 21945.Aug 5 2021, 10:22 PM

rebase

swh-public-ci added a comment.Aug 5 2021, 10:24 PM

Build is green

Patch application report for D6054 (id=21945)

Could not rebase; Attempt merge onto 509280132c...

Updating 5092801..f628703
Fast-forward
 mypy.ini                                           |   3 +
 requirements.txt                                   |   1 +
 setup.py                                           |   3 +-
 swh/provenance/api/serializers.py                  |  11 +-
 swh/provenance/api/server.py                       |  35 ++-
 swh/provenance/archive.py                          |   3 +
 swh/provenance/cli.py                              |   2 -
 swh/provenance/postgresql/archive.py               |   8 +-
 swh/provenance/postgresql/provenancedb_base.py     |   8 +-
 .../postgresql/provenancedb_with_path.py           |   8 +-
 .../postgresql/provenancedb_without_path.py        |  14 +-
 swh/provenance/storage/archive.py                  |   2 +-
 swh/provenance/tests/conftest.py                   | 223 +++------------
 swh/provenance/tests/data/README.md                |   2 +-
 swh/provenance/tests/data/generate_repo.py         |   9 +-
 .../tests/data/generate_storage_from_git.py        |  34 ++-
 .../data/origin-revision_with-merges_visits-01.txt |  67 +++++
 swh/provenance/tests/data/with-merges.msgpack      | Bin 7501 -> 10327 bytes
 swh/provenance/tests/test_archive_interface.py     |   4 +-
 swh/provenance/tests/test_cli.py                   |   5 +-
 swh/provenance/tests/test_conftest.py              |  18 +-
 swh/provenance/tests/test_history_graph.py         |   4 +-
 swh/provenance/tests/test_isochrone_graph.py       |   4 +-
 swh/provenance/tests/test_origin_iterator.py       |  36 +--
 swh/provenance/tests/test_origin_revision_layer.py | 190 +++++++++++++
 swh/provenance/tests/test_provenance_db.py         |  19 --
 swh/provenance/tests/test_provenance_storage.py    | 315 ++++++++++++++++++++-
 ...euristics.py => test_revision_content_layer.py} | 154 ++++++++--
 swh/provenance/tests/test_revision_iterator.py     |   6 +-
 29 files changed, 864 insertions(+), 324 deletions(-)
 create mode 100644 swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt
 create mode 100644 swh/provenance/tests/test_origin_revision_layer.py
 rename swh/provenance/tests/{test_provenance_heuristics.py => test_revision_content_layer.py} (70%)
Changes applied before test
commit f6287034e95bd4444a48bbd45019596e6db8a01e
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:37:39 2021 +0200

    Add test for the different `ProvenanceStorageInterface` implementations
    
    All possible implementations are instantiated in the `storage` fixture and tested
    against the reference on in `provenance. storage` fixture. The latter is in turn
    tested against the actual provenance algorithms.

commit f394b7a7396a86bd908044cd1c1755fac46d26e8
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:31:12 2021 +0200

    Make `provenance` fixture use a fixed storage backend for testing
    
    The parametrized `storage` fixture will be used to test all possible storage
    implementations against the single fixed one in `provenance.storage`.

commit 70651f45092ae590ee8a66febf4543cb37a0627f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 15:37:43 2021 +0200

    Refactor the use of archive `Storage` object for testing
    
    Remove any use of `swh.model.tests.swh_model_data.TEST_OBJECTS`, and keep a single fixture for
    `ArchiveInterface` objects based on `ArchiveStorage` (using `swh_storage` fixture as parameter).
    This avoids duplicating every test for each `ArchiveInterface` implementation. All
    implementations of `ArchiveInterface` are tested against each other in `test_archive_interface`.
    
    Also, `ArchiveStorage` now has a `storage` attribute to access the underlying `StorageInterface`
    object, so that test no longer needs to receive `archive` and `swh_storage` separately. Each
    test is responsible for filling `archive. storage` with the required data specific to the test.

commit b738c4610cb84f3f5d10a16f667e4522f7d49377
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit 16d91e21ff1429f897f953e122f6d21e2def0eb5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 92237ff7db1d74aa42c62b5bff4575f2ee0f4087
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/283/ for more details.

Harbormaster completed remote builds in B22908: Diff 21945.Aug 5 2021, 10:24 PM
aeviso updated this revision to Diff 21946.Aug 5 2021, 10:25 PM

rebase

swh-public-ci added a comment.Aug 5 2021, 10:28 PM

Build is green

Patch application report for D6054 (id=21946)

Could not rebase; Attempt merge onto 509280132c...

Updating 5092801..1c574dc
Fast-forward
 mypy.ini                                           |   3 +
 requirements.txt                                   |   1 +
 setup.py                                           |   3 +-
 swh/provenance/api/serializers.py                  |  11 +-
 swh/provenance/api/server.py                       |  35 ++-
 swh/provenance/archive.py                          |   3 +
 swh/provenance/cli.py                              |   2 -
 swh/provenance/postgresql/archive.py               |   8 +-
 swh/provenance/postgresql/provenancedb_base.py     |   8 +-
 .../postgresql/provenancedb_with_path.py           |   8 +-
 .../postgresql/provenancedb_without_path.py        |  14 +-
 swh/provenance/storage/archive.py                  |   2 +-
 swh/provenance/tests/conftest.py                   | 222 +++------------
 swh/provenance/tests/data/README.md                |   2 +-
 swh/provenance/tests/data/generate_repo.py         |   9 +-
 .../tests/data/generate_storage_from_git.py        |  34 ++-
 .../data/origin-revision_with-merges_visits-01.txt |  67 +++++
 swh/provenance/tests/data/with-merges.msgpack      | Bin 7501 -> 10327 bytes
 swh/provenance/tests/test_archive_interface.py     |   4 +-
 swh/provenance/tests/test_cli.py                   |   5 +-
 swh/provenance/tests/test_conftest.py              |  18 +-
 swh/provenance/tests/test_history_graph.py         |   4 +-
 swh/provenance/tests/test_isochrone_graph.py       |   4 +-
 swh/provenance/tests/test_origin_iterator.py       |  36 +--
 swh/provenance/tests/test_origin_revision_layer.py | 190 +++++++++++++
 swh/provenance/tests/test_provenance_db.py         |  19 --
 swh/provenance/tests/test_provenance_storage.py    | 315 ++++++++++++++++++++-
 ...euristics.py => test_revision_content_layer.py} | 154 ++++++++--
 swh/provenance/tests/test_revision_iterator.py     |   6 +-
 29 files changed, 863 insertions(+), 324 deletions(-)
 create mode 100644 swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt
 create mode 100644 swh/provenance/tests/test_origin_revision_layer.py
 rename swh/provenance/tests/{test_provenance_heuristics.py => test_revision_content_layer.py} (70%)
Changes applied before test
commit 1c574dc8faa7603bc61ee9fe5f20f0dbfa6b8e1f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:37:39 2021 +0200

    Add test for the different `ProvenanceStorageInterface` implementations
    
    All possible implementations are instantiated in the `storage` fixture and tested
    against the reference on in `provenance. storage` fixture. The latter is in turn
    tested against the actual provenance algorithms.

commit f394b7a7396a86bd908044cd1c1755fac46d26e8
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:31:12 2021 +0200

    Make `provenance` fixture use a fixed storage backend for testing
    
    The parametrized `storage` fixture will be used to test all possible storage
    implementations against the single fixed one in `provenance.storage`.

commit 70651f45092ae590ee8a66febf4543cb37a0627f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 15:37:43 2021 +0200

    Refactor the use of archive `Storage` object for testing
    
    Remove any use of `swh.model.tests.swh_model_data.TEST_OBJECTS`, and keep a single fixture for
    `ArchiveInterface` objects based on `ArchiveStorage` (using `swh_storage` fixture as parameter).
    This avoids duplicating every test for each `ArchiveInterface` implementation. All
    implementations of `ArchiveInterface` are tested against each other in `test_archive_interface`.
    
    Also, `ArchiveStorage` now has a `storage` attribute to access the underlying `StorageInterface`
    object, so that test no longer needs to receive `archive` and `swh_storage` separately. Each
    test is responsible for filling `archive. storage` with the required data specific to the test.

commit b738c4610cb84f3f5d10a16f667e4522f7d49377
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit 16d91e21ff1429f897f953e122f6d21e2def0eb5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 92237ff7db1d74aa42c62b5bff4575f2ee0f4087
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/284/ for more details.

Harbormaster completed remote builds in B22909: Diff 21946.Aug 5 2021, 10:28 PM
aeviso marked 2 inline comments as done.Aug 5 2021, 10:28 PM
douardda accepted this revision.Aug 6 2021, 10:59 AM
This revision is now accepted and ready to land.Aug 6 2021, 10:59 AM
aeviso updated this revision to Diff 21962.Aug 6 2021, 3:19 PM

rebase

swh-public-ci added a comment.Aug 6 2021, 3:39 PM

Build is green

Patch application report for D6054 (id=21962)

Could not rebase; Attempt merge onto 3b145f15c2...

Updating 3b145f1..980c560
Fast-forward
 mypy.ini                                           |   3 +
 requirements.txt                                   |   1 +
 setup.py                                           |   3 +-
 swh/provenance/api/serializers.py                  |  11 +-
 swh/provenance/api/server.py                       |  35 ++-
 swh/provenance/archive.py                          |   3 +
 swh/provenance/cli.py                              |   2 -
 swh/provenance/postgresql/archive.py               |   8 +-
 swh/provenance/postgresql/provenancedb.py          |  16 +-
 swh/provenance/storage/archive.py                  |   2 +-
 swh/provenance/tests/conftest.py                   | 229 +++------------
 swh/provenance/tests/data/README.md                |   2 +-
 swh/provenance/tests/data/generate_repo.py         |   9 +-
 .../tests/data/generate_storage_from_git.py        |  34 ++-
 .../data/origin-revision_with-merges_visits-01.txt |  67 +++++
 swh/provenance/tests/data/with-merges.msgpack      | Bin 7501 -> 10327 bytes
 swh/provenance/tests/test_archive_interface.py     |   4 +-
 swh/provenance/tests/test_cli.py                   |   5 +-
 swh/provenance/tests/test_conftest.py              |  18 +-
 swh/provenance/tests/test_history_graph.py         |   4 +-
 swh/provenance/tests/test_isochrone_graph.py       |  12 +-
 swh/provenance/tests/test_origin_iterator.py       |  36 +--
 swh/provenance/tests/test_origin_revision_layer.py | 190 +++++++++++++
 swh/provenance/tests/test_provenance_db.py         |  30 --
 swh/provenance/tests/test_provenance_storage.py    | 314 ++++++++++++++++++++-
 ...euristics.py => test_revision_content_layer.py} | 150 +++++++++-
 swh/provenance/tests/test_revision_iterator.py     |   9 +-
 27 files changed, 871 insertions(+), 326 deletions(-)
 create mode 100644 swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt
 create mode 100644 swh/provenance/tests/test_origin_revision_layer.py
 rename swh/provenance/tests/{test_provenance_heuristics.py => test_revision_content_layer.py} (71%)
Changes applied before test
commit 980c5605d9a568d1f61450161a00803147b90d60
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:37:39 2021 +0200

    Add test for the different `ProvenanceStorageInterface` implementations
    
    All possible implementations are instantiated in the `storage` fixture and tested
    against the reference on in `provenance. storage` fixture. The latter is in turn
    tested against the actual provenance algorithms.

commit 6f9f47cabb2082286cbfcbade196a6efbc108ddd
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:31:12 2021 +0200

    Make `provenance` fixture use a fixed storage backend for testing
    
    The parametrized `storage` fixture will be used to test all possible storage
    implementations against the single fixed one in `provenance.storage`.

commit d0a122af791ed2b07d1278bbc008d5531dff3023
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 15:37:43 2021 +0200

    Refactor the use of archive `Storage` object for testing
    
    Remove any use of `swh.model.tests.swh_model_data.TEST_OBJECTS`, and keep a single fixture for
    `ArchiveInterface` objects based on `ArchiveStorage` (using `swh_storage` fixture as parameter).
    This avoids duplicating every test for each `ArchiveInterface` implementation. All
    implementations of `ArchiveInterface` are tested against each other in `test_archive_interface`.
    
    Also, `ArchiveStorage` now has a `storage` attribute to access the underlying `StorageInterface`
    object, so that test no longer needs to receive `archive` and `swh_storage` separately. Each
    test is responsible for filling `archive. storage` with the required data specific to the test.

commit 91e4dcd4148da7763253f015e296dbc10ef2b564
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit be098b32d8fb808d447d888c573e731952bc12b0
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit c58d170f77661ddde409733462b7704705c5eefc
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/288/ for more details.

Harbormaster completed remote builds in B22925: Diff 21962.Aug 6 2021, 3:39 PM
aeviso updated this revision to Diff 21967.Aug 6 2021, 3:50 PM

rebase

swh-public-ci added a comment.Aug 6 2021, 3:58 PM

Build is green

Patch application report for D6054 (id=21967)

Could not rebase; Attempt merge onto 3b145f15c2...

Updating 3b145f1..3431de8
Fast-forward
 mypy.ini                                           |   3 +
 requirements.txt                                   |   1 +
 setup.py                                           |   3 +-
 swh/provenance/api/serializers.py                  |  11 +-
 swh/provenance/api/server.py                       |  35 ++-
 swh/provenance/archive.py                          |   3 +
 swh/provenance/cli.py                              |   2 -
 swh/provenance/postgresql/archive.py               |   8 +-
 swh/provenance/postgresql/provenancedb.py          |  17 +-
 swh/provenance/storage/archive.py                  |   2 +-
 swh/provenance/tests/conftest.py                   | 229 +++------------
 swh/provenance/tests/data/README.md                |   2 +-
 swh/provenance/tests/data/generate_repo.py         |   9 +-
 .../tests/data/generate_storage_from_git.py        |  34 ++-
 .../data/origin-revision_with-merges_visits-01.txt |  67 +++++
 swh/provenance/tests/data/with-merges.msgpack      | Bin 7501 -> 10327 bytes
 swh/provenance/tests/test_archive_interface.py     |   4 +-
 swh/provenance/tests/test_cli.py                   |   5 +-
 swh/provenance/tests/test_conftest.py              |  18 +-
 swh/provenance/tests/test_history_graph.py         |   4 +-
 swh/provenance/tests/test_isochrone_graph.py       |  12 +-
 swh/provenance/tests/test_origin_iterator.py       |  36 +--
 swh/provenance/tests/test_origin_revision_layer.py | 190 +++++++++++++
 swh/provenance/tests/test_provenance_db.py         |  30 --
 swh/provenance/tests/test_provenance_storage.py    | 314 ++++++++++++++++++++-
 ...euristics.py => test_revision_content_layer.py} | 150 +++++++++-
 swh/provenance/tests/test_revision_iterator.py     |   9 +-
 27 files changed, 872 insertions(+), 326 deletions(-)
 create mode 100644 swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt
 create mode 100644 swh/provenance/tests/test_origin_revision_layer.py
 rename swh/provenance/tests/{test_provenance_heuristics.py => test_revision_content_layer.py} (71%)
Changes applied before test
commit 3431de8f4931e5d202d65f57e8c04099de831911
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Aug 6 15:44:37 2021 +0200

    Add test for the different `ProvenanceStorageInterface` implementations
    
    All possible implementations are instantiated in the `storage` fixture and tested
    against the reference on in `provenance. storage` fixture. The latter is in turn
    tested against the actual provenance algorithms.

commit 0e10aa4c70987858a1a9549080708a79eb4e5bd5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Aug 6 15:43:11 2021 +0200

    Fix bug on `ProvenanceDB._relation_get` methods when `reverse=True`

commit ac6439aa7de24ffe0064b2051a32d1043aeda933
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Aug 3 16:31:12 2021 +0200

    Make `provenance` fixture use a fixed storage backend for testing
    
    The parametrized `storage` fixture will be used to test all possible storage
    implementations against the single fixed one in `provenance.storage`.

commit 6a207b0e6be02571acea3ca0d2f605d2721b4405
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 15:37:43 2021 +0200

    Refactor the use of archive `Storage` object for testing
    
    Remove any use of `swh.model.tests.swh_model_data.TEST_OBJECTS`, and keep a single fixture for
    `ArchiveInterface` objects based on `ArchiveStorage` (using `swh_storage` fixture as parameter).
    This avoids duplicating every test for each `ArchiveInterface` implementation. All
    implementations of `ArchiveInterface` are tested against each other in `test_archive_interface`.
    
    Also, `ArchiveStorage` now has a `storage` attribute to access the underlying `StorageInterface`
    object, so that test no longer needs to receive `archive` and `swh_storage` separately. Each
    test is responsible for filling `archive. storage` with the required data specific to the test.

commit 91e4dcd4148da7763253f015e296dbc10ef2b564
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit be098b32d8fb808d447d888c573e731952bc12b0
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit c58d170f77661ddde409733462b7704705c5eefc
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/290/ for more details.

Harbormaster completed remote builds in B22929: Diff 21967.Aug 6 2021, 3:58 PM
Closed by commit rDPROVac6439aa7de2: Make `provenance` fixture use a fixed storage backend for testing (authored by aeviso). · Explain WhyAug 6 2021, 4:00 PM
This revision was automatically updated to reflect the committed changes.
aeviso added a commit: rDPROVac6439aa7de2: Make `provenance` fixture use a fixed storage backend for testing.
aeviso added a commit: rDPROV0e10aa4c7098: Fix bug on `ProvenanceDB._relation_get` methods when `reverse=True`.
aeviso added a commit: rDPROV3431de8f4931: Add test for the different `ProvenanceStorageInterface` implementations.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
3 lines--
swh/
provenance/
postgresql/
5 lines90%
Loading...
tests/
36 lines98%
Loading...
312 lines94%
Loading...

Diff 21967

View Options

mypy.ini

Loading...
View Options

swh/provenance/postgresql/provenancedb.py

Loading...
View Options

swh/provenance/tests/conftest.py

Loading...
View Options

swh/provenance/tests/test_provenance_storage.py

Loading...
About · Developer information · Legal