Page MenuHomeSoftware Heritage
Differential D637 Diff 2066 swh/storage/storage.py

Changeset View

Standalone View

View Options

swh/storage/storage.py

Show First 20 Lines • Show All 1,366 Lines • ▼ Show 20 Lines def tool_get(self, tool, db=None, cur=None):
return dict(zip(db.tool_cols, idx)) return dict(zip(db.tool_cols, idx))
@db_transaction() @db_transaction()
def metadata_provider_add(self, provider_name, provider_type, provider_url, def metadata_provider_add(self, provider_name, provider_type, provider_url,
metadata, db=None, cur=None): metadata, db=None, cur=None):
"""Add a metadata provider. """Add a metadata provider.
Args: Args:
provider_name (str): Its name provider_name (str): Its name
moraneggUnsubmitted
Done
Inline Actions

It might be pertinent to write a type example or the potential types list.

moranegg: It might be pertinent to write a type example or the potential types list.
douarddaUnsubmitted
Not Done
Inline Actions

I'll go one step further:

  • what are the possible values for this provider_type (I've not checked the code for this) ? If its a limited list of possible values, give them, otherwise tell how valid values can be guessed.
  • add a complete and valid example of arguments (or give a clue on where such a usage example can be found in the code)

Also, there should somewhere a description of what this metadata object stands for. I guess it is the metadata describing the metadata provider (which might be source of confusion; which metadata are we talking about in there?)

douardda: I'll go one step further: - what are the possible values for this provider_type (I've not…
vlorentzAuthorUnsubmitted
Done
Inline Actions

tbh, I don't know the answers to these questions

vlorentz: tbh, I don't know the answers to these questions
moraneggUnsubmitted
Not Done
Inline Actions

what are the possible values for this provider_type (I've not checked the code for this) ? If its a limited list of possible values, give them, otherwise tell how valid values can be guessed.

It isn't a limited list for now, we have at the moment only one provider_type deposit_client, but the metadata provider entity should be used in the following future cases:

  • when listing: lister type
  • when loading: loader type
  • when fetching metadata from registries: registry type

Because it is not implemented, I'm not sure if we should list the types above in the docs

add a complete and valid example of arguments (or give a clue on where such a usage example can be found in the code)

Here is the actual metadata_provider in the storage today

idprovider_nameprovider_typeprovider_urlmetadata
3haldeposit_clienthttps://hal.archives-ouvertes.fr/{}
4swhdeposit_clienthttps://www.softwareheritage.org{}

with the use cases I mentioned we could add:

idprovider_nameprovider_typeprovider_urlmetadata
xgitlab_listerlisterhttps://gitlab.com/{}
ygitlab_loaderloaderhttps://gitlab.com/{}
zwikidataregistryhttps://www.wikidata.org{}

Also, there should somewhere a description of what this metadata object stands for. I guess it is the metadata describing the metadata provider (which might be source of confusion; which metadata are we talking about in there?)

You are right ! we do not use at the moment the metadata property for a metadata_provider, this property was added for registries where we want to keep more information about the registry itself.

@vlorentz you can add the existing hal deposit_client example as is
The rest isn't documented, we had a couple of emails on swh-devel, but we might need specs on this subject.

For a larger discussion:
@douardda where would you put specs on future features?
For the deposit, we used the \docs\specs for it, maybe wiki pages are more appropriate.

moranegg: > what are the possible values for this provider_type (I've not checked the code for this) ? If…
provider_type (str): Its type provider_type (str): Its type (eg. `'deposit-client'`)
provider_url (str): Its URL provider_url (str): Its URL
metadata: JSON-encodable object
Returns: Returns:
dict: same as args, plus an 'id' key. dict: same as args, plus an 'id' key.
""" """
return db.metadata_provider_add(provider_name, provider_type, return db.metadata_provider_add(provider_name, provider_type,
provider_url, metadata, cur) provider_url, metadata, cur)
@db_transaction() @db_transaction()
▲ Show 20 Lines • Show All 82 Lines • Show Last 20 Lines
About · Developer information · Legal