Remove the without-path flavor of ProvenanceDB
AbandonedPublic
Actions

Authored by douardda on Jun 10 2021, 10:35 AM.

Details

Reviewers

vlorentz
aeviso

Group Reviewers

Reviewers

Summary

Depends on D5829

Diff Detail

Repository

rDPROV Provenance database

Branch

denormalize

Lint

Lint Skipped

Unit

Unit Tests Skipped

Build Status

Buildable 21904
Build 34063: Phabricator diff pipeline on jenkins	Jenkins console · Jenkins
Build 34062: arc lint + arc unit

Event Timeline

douardda created this revision.Jun 10 2021, 10:35 AM

Herald added a reviewer: Reviewers. · View Herald TranscriptJun 10 2021, 10:35 AM

douardda added a child revision: D5842: Refactor the provenanceDB.insert_relation().Jun 10 2021, 10:37 AM

Build is green

Patch application report for D5841 (id=20894)

Could not rebase; Attempt merge onto 6cdd424eba...

Updating 6cdd424..fe35120
Fast-forward
 swh/provenance/__init__.py                         |  18 +-
 swh/provenance/postgresql/provenancedb.py          | 455 +++++++++++++++++++++
 swh/provenance/postgresql/provenancedb_base.py     | 325 ---------------
 .../postgresql/provenancedb_with_path.py           | 157 -------
 .../postgresql/provenancedb_without_path.py        | 140 -------
 swh/provenance/provenance.py                       |   3 +-
 swh/provenance/sql/15-flavor.sql                   |  21 -
 swh/provenance/sql/30-schema.sql                   |  25 +-
 swh/provenance/sql/60-indexes.sql                  |   7 -
 swh/provenance/tests/conftest.py                   |   4 +-
 swh/provenance/tests/test_cli.py                   |  33 +-
 11 files changed, 473 insertions(+), 715 deletions(-)
 create mode 100644 swh/provenance/postgresql/provenancedb.py
 delete mode 100644 swh/provenance/postgresql/provenancedb_base.py
 delete mode 100644 swh/provenance/postgresql/provenancedb_with_path.py
 delete mode 100644 swh/provenance/postgresql/provenancedb_without_path.py
 delete mode 100644 swh/provenance/sql/15-flavor.sql

Changes applied before test

commit fe35120741d76ff4d91d82bd1db029ff90ce8d60
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Jun 9 14:55:54 2021 +0200

    Remove the without-path flavor of ProvenanceDB

commit e23832b21ad4ee7afcb56f98147e51f633b6c2d7
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Jun 9 10:27:32 2021 +0200

    Refactor the cache handling in ProvenanceDB
    
    - use TypedDict structures to properly type the caches needed by the
      ProvenanceDB objects,
    - use only one cache plus a set of added (and eventually removed) ids of
      objects (within the cache) for revisisons, contents and directories.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/102/ for more details.

Harbormaster completed remote builds in B21877: Diff 20894.Jun 10 2021, 10:37 AM

douardda requested review of this revision.Jun 10 2021, 10:37 AM

I can't judge the substance, as usual, so just two nitpicks

swh/provenance/postgresql/provenancedb.py
15–16	can be replaced with https://docs.python.org/3/library/os.path.html#os.path.normpath
26–36	I find this slightly more readable, but feel free to ignore

This revision is now accepted and ready to land.Jun 10 2021, 10:55 AM

rebase

Build is green

Patch application report for D5841 (id=20922)

Could not rebase; Attempt merge onto 075b0d6cd6...

Updating 075b0d6..4c50588
Fast-forward
 swh/provenance/__init__.py                         |  18 +-
 swh/provenance/postgresql/provenancedb.py          | 455 +++++++++++++++++++++
 swh/provenance/postgresql/provenancedb_base.py     | 325 ---------------
 .../postgresql/provenancedb_with_path.py           | 157 -------
 .../postgresql/provenancedb_without_path.py        | 140 -------
 swh/provenance/provenance.py                       |   3 +-
 swh/provenance/sql/15-flavor.sql                   |  21 -
 swh/provenance/sql/30-schema.sql                   |  25 +-
 swh/provenance/sql/60-indexes.sql                  |   7 -
 swh/provenance/tests/conftest.py                   |   4 +-
 swh/provenance/tests/test_cli.py                   |  33 +-
 11 files changed, 473 insertions(+), 715 deletions(-)
 create mode 100644 swh/provenance/postgresql/provenancedb.py
 delete mode 100644 swh/provenance/postgresql/provenancedb_base.py
 delete mode 100644 swh/provenance/postgresql/provenancedb_with_path.py
 delete mode 100644 swh/provenance/postgresql/provenancedb_without_path.py
 delete mode 100644 swh/provenance/sql/15-flavor.sql

Changes applied before test

commit 4c50588e85be58c0d17d0e55d3ebb0facc3ee173
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Jun 9 14:55:54 2021 +0200

    Remove the without-path flavor of ProvenanceDB

commit 8aff35d251db39537a3a4bd14f98783dc06ebdc9
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Jun 9 10:27:32 2021 +0200

    Refactor the cache handling in ProvenanceDB
    
    - use TypedDict structures to properly type the caches needed by the
      ProvenanceDB objects,
    - use only one cache plus a set of added (and eventually removed) ids of
      objects (within the cache) for revisisons, contents and directories.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/114/ for more details.

Harbormaster completed remote builds in B21904: Diff 20922.Jun 10 2021, 2:56 PM

douardda removed a child revision: D5842: Refactor the provenanceDB.insert_relation().Jun 15 2021, 12:12 PM

aeviso requested changes to this revision.Jun 15 2021, 1:29 PM

aeviso added a subscriber: aeviso.

aeviso added inline comments.

swh/provenance/postgresql/provenancedb.py
64	This class is not supposed to handle cache anymore. It is now handled by the `ProvenanceBackend` (this name should be improved too) class in `provenance.py`. That's part of the ongoing refactoring.