Add Gitea/Gogs metadata fetcher
ClosedPublic
Actions

Authored by vlorentz on Aug 24 2022, 12:34 PM.

Details

Reviewers

douardda
anlambert

Group Reviewers

Reviewers

Maniphest Tasks

T4451: Fetch extrinsic metadata from Gitea/Gogs

Commits

rDLDMD7058293909dd: Add Gitea/Gogs metadata fetcher

Summary

Closes T4451.

Diff Detail

Repository

rDLDMD Extrinsic Metadata Loaders

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

Event Timeline

vlorentz created this revision.Aug 24 2022, 12:34 PM

Herald added a reviewer: Reviewers. · View Herald TranscriptAug 24 2022, 12:34 PM

Build has FAILED

Patch application report for D8303 (id=29977)

Rebasing onto 225d68c5b4...

Current branch diff-target is up to date.

Changes applied before test

commit 74d75fb25042ce66df0c25968eddcada736d4e1b
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Aug 24 12:34:09 2022 +0200

    Add Gitea/Gogs metadata fetcher

Link to build: https://jenkins.softwareheritage.org/job/DLDMD/job/tests-on-diff/12/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDMD/job/tests-on-diff/12/console

Harbormaster returned this revision to the author for changes because remote builds failed.Aug 24 2022, 12:34 PM

Harbormaster failed remote builds in B31026: Diff 29977!

remove file that shouldn't be committed

Build is green

Patch application report for D8303 (id=29978)

Rebasing onto 225d68c5b4...

Current branch diff-target is up to date.

Changes applied before test

commit c190a9ed8180becb59b1b4d5ba82369c39cb02b9
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Aug 24 12:34:09 2022 +0200

    Add Gitea/Gogs metadata fetcher

See https://jenkins.softwareheritage.org/job/DLDMD/job/tests-on-diff/13/ for more details.

Harbormaster completed remote builds in B31027: Diff 29978.Aug 24 2022, 12:40 PM

vlorentz requested review of this revision.Aug 24 2022, 12:40 PM

vlorentz added a parent revision: D8360: docs: Document metadata formats for Gogs and Gitea.Aug 31 2022, 2:58 PM

ok (but I have a couple of questions)

swh/loader/metadata/gitea.py
57	bit weird to have this hardcoded while there is the HTTP_ACCEPT "const" above
73	not gonna tell it's bad, but why do we enforce https only here?
85	there really should be a path/url normalization registry somewhere (not sure where) [unrelated with this diff]

This revision is now accepted and ready to land.Sep 2 2022, 11:19 AM

dedup constant + allow http scheme

Build is green

Patch application report for D8303 (id=30251)

Rebasing onto 225d68c5b4...

Current branch diff-target is up to date.

Changes applied before test

commit 7058293909ddc7d55d9fe3f21788ad3282faac11
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Aug 24 12:34:09 2022 +0200

    Add Gitea/Gogs metadata fetcher

See https://jenkins.softwareheritage.org/job/DLDMD/job/tests-on-diff/14/ for more details.

Harbormaster completed remote builds in B31289: Diff 30251.Sep 2 2022, 11:38 AM