Page MenuHomeSoftware Heritage

Deploy swh-loader-metadata v0.0.3
Closed, MigratedEdits Locked

Description

It adds metadata collection for Gitea.

(Reminder: swh-loader-metadata is a package that hooks into actual loaders; so the upgrade needs to happen on loaders)

Staging:

  • Configure Gitea credentials in loader configuration, to match listers if any (there should already be GitHub credentials, same as the listers)
  • Upgrade package on workers
  • Restart workers
  • Reset journal client on the swh.journal.objects.raw_extrinsic_metadata topic (to actually run the new SWORD metadata mapping)
  • Schedule a run (with lister_name=gitea and lister_instance_name set)
  • Make sure the metadata is loaded in swh-storage: select * from raw_extrinsic_metadata where target = 'swh:1:ori:...' and format = 'gitea-repository-json' limit 1;)

Production:

  • Configure Gitea credentials in loader configuration, to match listers if any (there should already be GitHub credentials, same as the listers)
  • Upgrade package on workers
  • Restart workers
  • so previous puppet changes can be applied
  • Reset journal client on the swh.journal.objects.raw_extrinsic_metadata topic (to actually run the new SWORD metadata mapping)
  • Schedule a run (with lister_name=gitea and lister_instance_name set)
  • Make sure the metadata is loaded in swh-storage: select * from raw_extrinsic_metadata where target = 'swh:1:ori:...' and format = 'gitea-repository-json' limit 1;)

Event Timeline

vlorentz triaged this task as Normal priority.Oct 5 2022, 1:04 PM
vlorentz created this task.
vlorentz updated the task description. (Show Details)
vlorentz updated the task description. (Show Details)

also, this should preferably be deployed after swh-indexer v2.7.0 so we don't need to reset indexer journal consumers to index metadata from Gitea, but not a requirement.

vlorentz renamed this task from Deploy swh-loader-metadata v0.0.3 on production and staging to Deploy swh-loader-metadata v0.0.3.Oct 5 2022, 1:39 PM

Configure Gitea credentials

jsyk, we don't have any gitea credentials.