Page MenuHomeSoftware Heritage

Fix heterogeneity of names in metadata tables

Authored by haltode on Mar 7 2019, 8:16 AM.



Fixes T1561:

  • revision_metadata renamed to revision_intrinsic_metadata,
  • revision_metadata.translated_metadata renamed to revision_metadata.metadata,
  • content_metadata.translated_metadata renamed to content_metadata.metadata,
  • origin_intrinsic_metadata.origin_id renamed to
  • rename/update functions and affected tests
  • add database migration script

Diff Detail

rDCIDX Metadata indexer
No Linters Available
No Unit Test Coverage
Build Status
Buildable 4553
Build 6042: tox-on-jenkinsJenkins
Build 6041: arc lint + arc unit

Event Timeline

haltode created this revision.Mar 7 2019, 8:16 AM

all of which because of the same error).

Looks like Jenkins could not apply your patch. You should rebase your changes on the latest master and arc diff --update D1226

haltode updated this revision to Diff 3842.Mar 7 2019, 9:46 AM

Rebasing on master

haltode updated this revision to Diff 3890.EditedMar 12 2019, 5:06 AM

Fix failing tests and rename content_metadata.translated_metadata column.

haltode retitled this revision from [WIP][DO NOT MERGE] Fix heterogeneity of names in metadata tables to Fix heterogeneity of names in metadata tables.Mar 12 2019, 5:08 AM
haltode edited the summary of this revision. (Show Details)

Cool, thanks!

There are some little things to change (see inline comments), but other than that it's good!


You should add migrations for the functions as well (just copy-paste the affected create or replace function statements).


You can change _check_duplicates to remove the second argument, it's no longer needed now.


Can be rewritten as a generator expression, which I find more readable:

keys = (convert_key(col, 'oim') for col in self.origin_intrinsic_metadata_cols)

(same below)


debugging leftover? ^^


That line (and other similar ones) can be dropped now

haltode updated this revision to Diff 3893.Mar 12 2019, 12:09 PM

Updating D1226 concerning comments

vlorentz added inline comments.Mar 12 2019, 12:25 PM

The code of these functions must be updated as well


Sorry, one more nitpick: the for loop can be replaced by: yield from self._origin_intrinsic_metadata.get(ids)

haltode updated this revision to Diff 3894.Mar 12 2019, 3:41 PM
haltode marked 6 inline comments as done.

Update functions code during migration + More pythonic yield statement

Migration of swh_content_metadata_add and swh_origin_intrinsic_metadata_add are still missing. Oh, and I just noticed that the name change of index revision_intrinsic_metadata_pkey is also missing

haltode updated this revision to Diff 3897.Mar 13 2019, 3:38 AM
haltode marked 2 inline comments as done.

Add missing functions/index migrations.

haltode edited the summary of this revision. (Show Details)Mar 13 2019, 4:47 AM
vlorentz accepted this revision.Mar 13 2019, 9:48 AM
This revision is now accepted and ready to land.Mar 13 2019, 9:48 AM
haltode updated this revision to Diff 3907.Mar 13 2019, 1:21 PM

Rebasing on master

This revision was automatically updated to reflect the committed changes.