Page MenuHomeSoftware Heritage

Improve DB schema naming
ClosedPublic

Authored by douardda on Jun 15 2021, 12:11 PM.

Details

Summary

make it more consistant, so we can then simplify the code a bit by
making relation table names composable (i.e. the relation table between
content and revision is now simply content_in_revision, and so on) as
well as column names (e.g. rev->revision and so on).

Depends on D5842.

Diff Detail

Repository
rDPROV Provenance database
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build is green

Patch application report for D5870 (id=21023)

Could not rebase; Attempt merge onto 206399eb8a...

Updating 206399e..3e44e5e
Fast-forward
 swh/provenance/postgresql/provenancedb_base.py     |  66 +++++----
 .../postgresql/provenancedb_with_path.py           |  89 +++++-------
 .../postgresql/provenancedb_without_path.py        | 129 +++++++-----------
 swh/provenance/provenance.py                       | 104 +++++++-------
 swh/provenance/sql/30-schema.sql                   | 149 +++++++++------------
 swh/provenance/sql/60-indexes.sql                  |  17 +--
 swh/provenance/tests/test_cli.py                   |  11 +-
 swh/provenance/tests/test_provenance_heuristics.py |  84 +++++-------
 8 files changed, 278 insertions(+), 371 deletions(-)
Changes applied before test
commit 3e44e5ef68dae712bd248559d92f45a80dc7a4f2
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Jun 15 12:01:15 2021 +0200

    Improve DB schema naming
    
    make it more consistant, so we can then simplify the code a bit by
    making relation table names composable (i.e. the relation table between
    content and revision is now simply content_in_revision, and so on) as
    well as column names (e.g. rev->revision and so on).

commit c4eaa2d6c8f8762d20921c97d0ca01b54d5d81fa
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 14 16:01:24 2021 +0200

    Refactor the provenanceDB.insert_relation() methods
    
    simplify the code and reduce it to a couple of INSERT queries (one for
    locations if any, one for the destination relation table).

commit 29673033e93e423ccd39cca46731c57faa22b02a
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Jun 9 10:27:32 2021 +0200

    Refactor the cache handling in ProvenanceDB
    
    - use TypedDict structures to properly type the caches needed by the
      ProvenanceDB objects,
    - use only one (sha1, date) cache per entity, plus a set of added ids of
      objects (within the cache) (i.e. for revisisons, contents and directories).

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/134/ for more details.

Build is green

Patch application report for D5870 (id=21037)

Could not rebase; Attempt merge onto 8c536b8d50...

Updating 8c536b8..737c6e6
Fast-forward
 swh/provenance/postgresql/provenancedb_base.py     |  66 +++++----
 .../postgresql/provenancedb_with_path.py           |  89 +++++-------
 .../postgresql/provenancedb_without_path.py        | 129 +++++++-----------
 swh/provenance/provenance.py                       | 104 +++++++-------
 swh/provenance/sql/30-schema.sql                   | 149 +++++++++------------
 swh/provenance/sql/60-indexes.sql                  |  17 +--
 swh/provenance/tests/test_cli.py                   |  11 +-
 swh/provenance/tests/test_provenance_heuristics.py |  84 +++++-------
 8 files changed, 278 insertions(+), 371 deletions(-)
Changes applied before test
commit 737c6e662292f893e07dd8383d61ff5099d44e07
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Jun 15 12:01:15 2021 +0200

    Improve DB schema naming
    
    make it more consistant, so we can then simplify the code a bit by
    making relation table names composable (i.e. the relation table between
    content and revision is now simply content_in_revision, and so on) as
    well as column names (e.g. rev->revision and so on).

commit e27c7d11f2ce8e979c56cb49c75d0f0940993181
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 14 16:01:24 2021 +0200

    Refactor the provenanceDB.insert_relation() methods
    
    simplify the code and reduce it to a couple of INSERT queries (one for
    locations if any, one for the destination relation table).

commit ac8dc036bcc608ba39d65929569f13ad694ebd90
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Jun 9 10:27:32 2021 +0200

    Refactor the cache handling in ProvenanceDB
    
    - use TypedDict structures to properly type the caches needed by the
      ProvenanceDB objects,
    - use only one (sha1, date) cache per entity, plus a set of added ids of
      objects (within the cache) (i.e. for revisisons, contents and directories).

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/138/ for more details.

Build is green

Patch application report for D5870 (id=21039)

Could not rebase; Attempt merge onto 8c536b8d50...

Updating 8c536b8..8d23dac
Fast-forward
 swh/provenance/postgresql/provenancedb_base.py     |  66 +++++----
 .../postgresql/provenancedb_with_path.py           |  89 +++++-------
 .../postgresql/provenancedb_without_path.py        | 129 +++++++-----------
 swh/provenance/provenance.py                       | 104 +++++++-------
 swh/provenance/sql/30-schema.sql                   | 149 +++++++++------------
 swh/provenance/sql/60-indexes.sql                  |  17 +--
 swh/provenance/tests/test_cli.py                   |  11 +-
 swh/provenance/tests/test_provenance_heuristics.py |  84 +++++-------
 8 files changed, 278 insertions(+), 371 deletions(-)
Changes applied before test
commit 8d23dac058f68ae05cc4cbcbd23a512a43723331
Author: David Douard <david.douard@sdfa3.org>
Date:   Tue Jun 15 12:01:15 2021 +0200

    Improve DB schema naming
    
    make it more consistent, so we can then simplify the code a bit by
    making relation table names composable (i.e. the relation table between
    content and revision is now simply content_in_revision, and so on) as
    well as column names (e.g. rev->revision and so on).
    
    Also remove the conditional sql execution in db init code (in
    swh/provennce/sql/), because it is actually unnecessary: keeping the
    `location` column for the "without-path" flavor does not cost anything.
    The only regression is that the location column in entity tables lost
    its "not null" constraint, but it's a minor drawback that the gain in
    clarity and simplicity of the db initialization code make up.

commit e27c7d11f2ce8e979c56cb49c75d0f0940993181
Author: David Douard <david.douard@sdfa3.org>
Date:   Mon Jun 14 16:01:24 2021 +0200

    Refactor the provenanceDB.insert_relation() methods
    
    simplify the code and reduce it to a couple of INSERT queries (one for
    locations if any, one for the destination relation table).

commit ac8dc036bcc608ba39d65929569f13ad694ebd90
Author: David Douard <david.douard@sdfa3.org>
Date:   Wed Jun 9 10:27:32 2021 +0200

    Refactor the cache handling in ProvenanceDB
    
    - use TypedDict structures to properly type the caches needed by the
      ProvenanceDB objects,
    - use only one (sha1, date) cache per entity, plus a set of added ids of
      objects (within the cache) (i.e. for revisisons, contents and directories).

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/139/ for more details.

This revision is now accepted and ready to land.Jun 15 2021, 3:53 PM
This revision was automatically updated to reflect the committed changes.