Details

Reviewers

Group Reviewers

Commits

rDPROV91e4dcd4148d: Add some missing type annotations and remove already done TODO's
rDPROVbe098b32d8fb: Add test for origin-revision layer
rDPROVc58d170f7766: Fix type conversion when retrieving origins' urls from PostgreSQL backend

Summary

Rename revision-content layer test as well, for consistency.

Depends on D5985

Diff Detail

Repository

rDPROV Provenance database

Branch

master

Lint

Lint Skipped

Unit

Unit Tests Skipped

Build Status

Buildable 22753
Build 35483: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 35482: arc lint + arc unit

Event Timeline

aeviso created this revision.Jul 27 2021, 12:40 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptJul 27 2021, 12:40 PM

Build is green

Patch application report for D6026 (id=21785)

Could not rebase; Attempt merge onto 1ae32c0a61...

Updating 1ae32c0..1497b12
Fast-forward
 swh/provenance/graph.py                            |  19 +-
 swh/provenance/origin.py                           |  19 +-
 swh/provenance/postgresql/provenancedb_base.py     |   4 +-
 swh/provenance/tests/conftest.py                   | 128 +-------------
 swh/provenance/tests/data/README.md                |   2 +-
 swh/provenance/tests/data/generate_repo.py         |   2 +-
 .../data/history_graphs_with-merges_visits-01.yaml | 144 +++++++++++++++-
 .../data/origin-revision_with-merges_visits-01.txt |  67 +++++++
 swh/provenance/tests/data/with-merges.msgpack      | Bin 7501 -> 10327 bytes
 swh/provenance/tests/test_origin_revision_layer.py | 192 +++++++++++++++++++++
 ...euristics.py => test_revision_content_layer.py} | 144 ++++++++++++++--
 11 files changed, 555 insertions(+), 166 deletions(-)
 create mode 100644 swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt
 create mode 100644 swh/provenance/tests/test_origin_revision_layer.py
 rename swh/provenance/tests/{test_provenance_heuristics.py => test_revision_content_layer.py} (71%)

Changes applied before test

commit 1497b12545e3dd62125333e625d0893b75648f34
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Tue Jul 27 12:35:48 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 509280132cb35df1432905097dafb24870690ee1
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 7 12:07:07 2021 +0200

    Simplify history graph creation and origin-revision algorithm

commit 505869054bda044c72d1a907631db552d173870f
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 7 10:59:12 2021 +0200

    Add missing case to history graph yaml test file
    
    Fix typo too

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/267/ for more details.

Harbormaster completed remote builds in B22753: Diff 21785.Jul 27 2021, 12:46 PM

aeviso requested review of this revision.Jul 27 2021, 12:46 PM

I am not fond at all of the code duplication (between R-C and O-R synth file parsers), looks to me at least parts of it could be kept factorised in a dedicated module (I agree it should not live in conftest any more: too much code and logic now). It would then be best to have these test-helper functions tested themselves (as unitary as possible).

In any case this revision needs to be split at least in 2 revisions.

swh/provenance/postgresql/provenancedb_base.py
113	This is not related to adding a test, it's a fix of the business code. So either it should be in a dedicated revision or the git commit message should be updated to something like "Fix the origin_get() method [...]" and mention that a new test is added to validate the fix.

This revision now requires changes to proceed.Jul 27 2021, 6:11 PM

aeviso marked an inline comment as done.Jul 28 2021, 11:31 AM

rebase

Build has FAILED

Patch application report for D6026 (id=21824)

Rebasing onto 509280132c...

Current branch diff-target is up to date.

Changes applied before test

commit f4ee11ace0d8a8648e9d8bec650bbf2292ed8454
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 92237ff7db1d74aa42c62b5bff4575f2ee0f4087
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

Link to build: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/271/
See console output for more information: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/271/console

Harbormaster failed remote builds in B22788: Diff 21824!Jul 28 2021, 11:38 AM

rebase

Build is green

Patch application report for D6026 (id=21867)

Rebasing onto 509280132c...

Current branch diff-target is up to date.

Changes applied before test

commit 16d91e21ff1429f897f953e122f6d21e2def0eb5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 92237ff7db1d74aa42c62b5bff4575f2ee0f4087
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/278/ for more details.

Harbormaster completed remote builds in B22828: Diff 21867.Jul 30 2021, 11:26 AM

rebase

Build is green

Patch application report for D6026 (id=21880)

Rebasing onto 509280132c...

Current branch diff-target is up to date.

Changes applied before test

commit 185d5b444625e5e603b8c9ae69a421398a950164
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit 16d91e21ff1429f897f953e122f6d21e2def0eb5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 92237ff7db1d74aa42c62b5bff4575f2ee0f4087
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/279/ for more details.

Harbormaster completed remote builds in B22843: Diff 21880.Aug 2 2021, 2:03 PM

rebase

aeviso added a child revision: D6053: Refactor the use of archive `Storage` object for testing.Aug 3 2021, 4:44 PM

Build is green

Patch application report for D6026 (id=21904)

Rebasing onto 509280132c...

Current branch diff-target is up to date.

Changes applied before test

commit b738c4610cb84f3f5d10a16f667e4522f7d49377
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit 16d91e21ff1429f897f953e122f6d21e2def0eb5
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit 92237ff7db1d74aa42c62b5bff4575f2ee0f4087
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/280/ for more details.

Harbormaster completed remote builds in B22870: Diff 21904.Aug 3 2021, 4:50 PM

douardda accepted this revision.Aug 5 2021, 12:08 PM

This revision is now accepted and ready to land.Aug 5 2021, 12:08 PM

rebae

Build is green

Patch application report for D6026 (id=21960)

Rebasing onto 3b145f15c2...

Current branch diff-target is up to date.

Changes applied before test

commit 91e4dcd4148da7763253f015e296dbc10ef2b564
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Fri Jul 30 13:46:49 2021 +0200

    Add some missing type annotations and remove already done TODO's

commit be098b32d8fb808d447d888c573e731952bc12b0
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:15:03 2021 +0200

    Add test for origin-revision layer
    
    Rename revision-content layer test as well, for consistency.

commit c58d170f77661ddde409733462b7704705c5eefc
Author: Andres Ezequiel Viso <aeviso@softwareheritage.org>
Date:   Wed Jul 28 11:11:49 2021 +0200

    Fix type conversion when retrieving origins' urls from PostgreSQL backend
    
    Correct some SQL code capitalization as well.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/286/ for more details.

Harbormaster completed remote builds in B22923: Diff 21960.Aug 6 2021, 3:30 PM

Closed by commit rDPROVc58d170f7766: Fix type conversion when retrieving origins' urls from PostgreSQL backend (authored by aeviso). · Explain WhyAug 6 2021, 4:00 PM

This revision was automatically updated to reflect the committed changes.

aeviso added a commit: rDPROVc58d170f7766: Fix type conversion when retrieving origins' urls from PostgreSQL backend.

aeviso added a commit: rDPROVbe098b32d8fb: Add test for origin-revision layer.

aeviso added a commit: rDPROV91e4dcd4148d: Add some missing type annotations and remove already done TODO's.

Add test for origin-revision layer
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Patch application report for D6026 (id=21785)

Changes applied before test

Patch application report for D6026 (id=21824)

Changes applied before test

Patch application report for D6026 (id=21867)

Changes applied before test

Patch application report for D6026 (id=21880)

Changes applied before test

Patch application report for D6026 (id=21904)

Changes applied before test

Patch application report for D6026 (id=21960)

Changes applied before test

Revision Contents
Changeset List

Diff 21785

swh/provenance/postgresql/provenancedb_base.py

swh/provenance/tests/conftest.py

swh/provenance/tests/data/README.md

swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt

swh/provenance/tests/data/with-merges.msgpack

swh/provenance/tests/test_origin_revision_layer.py

swh/provenance/tests/test_provenance_heuristics.py

swh/provenance/tests/test_revision_content_layer.py

Add test for origin-revision layerClosedPublicActions

Details

Diff Detail

Event Timeline

Patch application report for D6026 (id=21785)

Changes applied before test

Patch application report for D6026 (id=21824)

Changes applied before test

Patch application report for D6026 (id=21867)

Changes applied before test

Patch application report for D6026 (id=21880)

Changes applied before test

Patch application report for D6026 (id=21904)

Changes applied before test

Patch application report for D6026 (id=21960)

Changes applied before test

Revision ContentsChangeset List

Diff 21785

swh/provenance/postgresql/provenancedb_base.py

swh/provenance/tests/conftest.py

swh/provenance/tests/data/README.md

swh/provenance/tests/data/origin-revision_with-merges_visits-01.txt

swh/provenance/tests/data/with-merges.msgpack

swh/provenance/tests/test_origin_revision_layer.py

swh/provenance/tests/test_provenance_heuristics.py

swh/provenance/tests/test_revision_content_layer.py

Add test for origin-revision layer
ClosedPublic
Actions

Revision Contents
Changeset List