Paths

Table of Contentst

Add ExtID query support to the Storage
b8e10f00cfbc
Actions

Description

Add ExtID query support to the Storage

These endpoints allow to add and query the storage for known ExtID from SWHID
(typically get original VCS' revision intrinsic identifier from SWHID).

The underlying data structure is to be filled typically by loaders using
the extid_add() endpoint.

This only provides the Postgresql implementation.

Related to T2849.

Details

Provenance

douardda	Authored on Dec 9 2020, 4:57 PM
douardda	Pushed on Mar 11 2021, 2:54 PM

Differential Revision

D4698: Add support for ExtID in the storage

Parents

rDSTO6a7773225d41: Add hg revisions to the test data set

Branches

Unknown

Tags

Unknown

Tasks

T2849: Design and implement a mapping from "original VCS ids" to SWHIDs to help incremental loaders

Build Status

Buildable 19843
Build 30814: test-and-build	Jenkins console · Jenkins

Event Timeline

douardda committed rDSTOb8e10f00cfbc: Add ExtID query support to the Storage (authored by douardda).Mar 11 2021, 2:20 PM

douardda added a task: T2849: Design and implement a mapping from "original VCS ids" to SWHIDs to help incremental loaders.Mar 11 2021, 2:54 PM

douardda added an edge: D4698: Add support for ExtID in the storage.

Harbormaster completed building B19843: rDSTOb8e10f00cfbc: Add ExtID query support to the Storage.Mar 11 2021, 3:11 PM

swh-public-ci mentioned this in D4615: storage: Add branch names filtering support in snapshot_get_branches.Mar 11 2021, 3:13 PM

swh-public-ci mentioned this in D5208: storage: Allow to filter out branches by prefix when counting them.Mar 11 2021, 3:57 PM

Changes (24)

Path

Size

requirements-swh.txt

sql/

upgrades/

168.sql

swh/

storage/

api/

cassandra/

postgresql/

sql/

tests/

rDSTOb8e10f00cfbc

View Options

requirements-swh.txt

View Options

sql/upgrades/168.sql

View Options

swh/storage/api/serializers.py

View Options

swh/storage/backfill.py

View Options

swh/storage/cassandra/cql.py

View Options

swh/storage/cassandra/model.py

View Options

swh/storage/cassandra/schema.py

View Options

swh/storage/cassandra/storage.py

View Options

swh/storage/in_memory.py

View Options

swh/storage/interface.py

View Options

swh/storage/postgresql/converters.py

View Options

swh/storage/postgresql/db.py

View Options

swh/storage/postgresql/storage.py

View Options

swh/storage/replay.py

View Options

swh/storage/sql/30-schema.sql

View Options

swh/storage/sql/40-funcs.sql

View Options

swh/storage/sql/60-indexes.sql

View Options

swh/storage/tests/storage_data.py

View Options

swh/storage/tests/storage_tests.py

View Options

swh/storage/tests/test_backfill.py

View Options

swh/storage/tests/test_cassandra.py

View Options

swh/storage/tests/test_kafka_writer.py

View Options

swh/storage/tests/test_replay.py

View Options

Add ExtID query support to the Storageb8e10f00cfbcActions

Description

Details

Event Timeline

Changes (24)

rDSTOb8e10f00cfbc

requirements-swh.txt

sql/upgrades/168.sql

swh/storage/api/serializers.py

swh/storage/backfill.py

swh/storage/cassandra/cql.py

swh/storage/cassandra/model.py

swh/storage/cassandra/schema.py

swh/storage/cassandra/storage.py

swh/storage/in_memory.py

swh/storage/interface.py

swh/storage/postgresql/converters.py

swh/storage/postgresql/db.py

swh/storage/postgresql/storage.py

swh/storage/replay.py

swh/storage/sql/30-schema.sql

swh/storage/sql/40-funcs.sql

swh/storage/sql/60-indexes.sql

swh/storage/tests/storage_data.py

swh/storage/tests/storage_tests.py

swh/storage/tests/test_backfill.py

swh/storage/tests/test_cassandra.py

swh/storage/tests/test_kafka_writer.py

swh/storage/tests/test_replay.py

swh/storage/writer.py

Add ExtID query support to the Storage
b8e10f00cfbc
Actions