Page MenuHomeSoftware Heritage

Fix heterogeneity of names in metadata tables
ClosedPublic

Authored by haltode on Mar 7 2019, 8:16 AM.

Details

Summary

Fixes T1561:

  • revision_metadata renamed to revision_intrinsic_metadata,
  • revision_metadata.translated_metadata renamed to revision_metadata.metadata,
  • content_metadata.translated_metadata renamed to content_metadata.metadata,
  • origin_intrinsic_metadata.origin_id renamed to origin_intrinsic_metadata.id.
  • rename/update functions and affected tests
  • add database migration script

Diff Detail

Repository
rDCIDX Metadata indexer
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

all of which because of the same error).

Looks like Jenkins could not apply your patch. You should rebase your changes on the latest master and arc diff --update D1226

Fix failing tests and rename content_metadata.translated_metadata column.

haltode retitled this revision from [WIP][DO NOT MERGE] Fix heterogeneity of names in metadata tables to Fix heterogeneity of names in metadata tables.Mar 12 2019, 5:08 AM
haltode edited the summary of this revision. (Show Details)

Cool, thanks!

There are some little things to change (see inline comments), but other than that it's good!

sql/upgrades/123.sql
13

You should add migrations for the functions as well (just copy-paste the affected create or replace function statements).

swh/indexer/storage/__init__.py
726–727

You can change _check_duplicates to remove the second argument, it's no longer needed now.

swh/indexer/storage/db.py
380–383

Can be rewritten as a generator expression, which I find more readable:

keys = (convert_key(col, 'oim') for col in self.origin_intrinsic_metadata_cols)

(same below)

381

debugging leftover? ^^

swh/indexer/storage/in_memory.py
620–621

That line (and other similar ones) can be dropped now

Updating D1226 concerning comments

sql/upgrades/123.sql
9–10

The code of these functions must be updated as well

swh/indexer/storage/in_memory.py
620–621

Sorry, one more nitpick: the for loop can be replaced by: yield from self._origin_intrinsic_metadata.get(ids)

haltode marked 6 inline comments as done.

Update functions code during migration + More pythonic yield statement

Migration of swh_content_metadata_add and swh_origin_intrinsic_metadata_add are still missing. Oh, and I just noticed that the name change of index revision_intrinsic_metadata_pkey is also missing

haltode marked 2 inline comments as done.

Add missing functions/index migrations.

This revision is now accepted and ready to land.Mar 13 2019, 9:48 AM
This revision was automatically updated to reflect the committed changes.