Page MenuHomeSoftware Heritage
Differential D8160

gogs: Introduce Gogs lister
ClosedPublic
Actions

Authored by KShivendu on Jul 23 2022, 6:08 AM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T1721: Implementation of Gogs Lister
Commits
rDLSd34a6232a693: gogs: Introduce Gogs lister
Summary

Add lister for Gogs instances

Related to T1721

Diff Detail

Repository
rDLS Listers
Branch
feat/gogs-lister
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 30543
Build 47757: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 47756: arc lint + arc unit

Event Timeline

KShivendu created this revision.Jul 23 2022, 6:08 AM
Herald added a reviewer: Reviewers. · View Herald TranscriptJul 23 2022, 6:08 AM
swh-public-ci added a comment.Jul 23 2022, 6:12 AM

Build is green

Patch application report for D8160 (id=29468)

Rebasing onto 1bf11aa26d...

Current branch diff-target is up to date.
Changes applied before test
commit c8e83162ec1640572846c65d664d2dafea85474b
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 09:35:33 2022 +0530

    gogs: Introduce Gogs lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/564/ for more details.

Harbormaster completed remote builds in B30543: Diff 29468.Jul 23 2022, 6:12 AM
KShivendu requested review of this revision.Jul 23 2022, 6:12 AM
KShivendu updated this revision to Diff 29469.Jul 23 2022, 6:37 AM
  • gogs: Improve docstrings and comments
swh-public-ci added a comment.Jul 23 2022, 6:41 AM

Build is green

Patch application report for D8160 (id=29469)

Rebasing onto 1bf11aa26d...

Current branch diff-target is up to date.
Changes applied before test
commit 0755827d3d2d41454b74da377149566b7ef0433c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 10:04:56 2022 +0530

    gogs: Improve docstrings and comments

commit c8e83162ec1640572846c65d664d2dafea85474b
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 09:35:33 2022 +0530

    gogs: Introduce Gogs lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/565/ for more details.

Harbormaster completed remote builds in B30544: Diff 29469.Jul 23 2022, 6:41 AM
KShivendu updated this revision to Diff 29470.Jul 23 2022, 7:01 AM
  • gogs: Add more tests to increase coverage
swh-public-ci added a comment.Jul 23 2022, 7:05 AM

Build is green

Patch application report for D8160 (id=29470)

Rebasing onto 1bf11aa26d...

Current branch diff-target is up to date.
Changes applied before test
commit af49f7f36b672f59557dfda6450c274b32eba4f3
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 10:30:20 2022 +0530

    gogs: Add more tests to increase coverage

commit 0755827d3d2d41454b74da377149566b7ef0433c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 10:04:56 2022 +0530

    gogs: Improve docstrings and comments

commit c8e83162ec1640572846c65d664d2dafea85474b
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 09:35:33 2022 +0530

    gogs: Introduce Gogs lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/566/ for more details.

Harbormaster completed remote builds in B30545: Diff 29470.Jul 23 2022, 7:05 AM
ardumont added a subscriber: ardumont.Jul 25 2022, 9:43 AM

Hello, thx for the work. I did not check yet but can you please add the Related to TXYZ with TXYZ the related task id in the diff description?
TIA

ardumont requested changes to this revision.Jul 26 2022, 12:49 PM

@KShivendu A first round of review, I did not check the tests but this seems to do the job nicely according to the coverage.

Can you also please attend to the pagination link and have a look at D8160#212288 ?

Thanks in advance

swh/lister/gogs/__init__.py
1

missing copyright header, please add them.

6

we can use f-strings for that now.

swh/lister/gogs/lister.py
1
24
120

Can you please check whether the response has some pagination link inside?

It seems the documentation mentions it [1]
It'd be best to use pagination link instead of crafting the next page ourselves.
It will be more resilient over time.
And we even have some listers which lift this [2] ;).

[1] https://github.com/gogs/docs-api#link-header

[2] https://forge.softwareheritage.org/source/swh-lister/browse/master/swh/lister/gitea/lister.py$116-128

swh/lister/gogs/tasks.py
1
This revision now requires changes to proceed.Jul 26 2022, 12:49 PM
ardumont edited the summary of this revision. (Show Details)Jul 29 2022, 2:43 PM
ardumont added a task: T1721: Implementation of Gogs Lister.

Can you also please attend to the pagination link and have a look at D8160#212288 ?

I did it since you never replied and i need to summarize stuff.
But please, do mind it for next time (that helps).

Cheers,

KShivendu updated this revision to Diff 29500.Aug 1 2022, 4:22 PM
KShivendu marked 5 inline comments as done.
  • Use request's link header for pagination and update relevant tests
  • Fix copyright year and add missing copyright comments
  • Update comments wherever suggested
swh-public-ci added a comment.Aug 1 2022, 4:27 PM

Build is green

Patch application report for D8160 (id=29500)

Rebasing onto 1bf11aa26d...

Current branch diff-target is up to date.
Changes applied before test
commit 8fabff75390e5795ebe2f8f561ce5165364741ae
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Mon Aug 1 19:48:11 2022 +0530

    fix: Changes suggested in review
    
    - Use request's link header for pagination and update relevant tests
    - Fix copyright year and add missing copyright comments
    - Update comments wherever suggested

commit af49f7f36b672f59557dfda6450c274b32eba4f3
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 10:30:20 2022 +0530

    gogs: Add more tests to increase coverage

commit 0755827d3d2d41454b74da377149566b7ef0433c
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 10:04:56 2022 +0530

    gogs: Improve docstrings and comments

commit c8e83162ec1640572846c65d664d2dafea85474b
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 09:35:33 2022 +0530

    gogs: Introduce Gogs lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/568/ for more details.

Harbormaster completed remote builds in B30570: Diff 29500.Aug 1 2022, 4:27 PM
KShivendu added a comment.EditedAug 1 2022, 4:49 PM

Thanks for your feedback @ardumont

But please, do mind it for next time

For sure. Today, I've officially started my work with swh. I'll be proactive now ;)

Few things:

  • I have discovered an issue in the Gogs API. Please see T1721#88798
  • There are tons of similarities between Gitea and Gogs. Should we consider sharing code between the two?

Cheers!

ardumont added a comment.Aug 2 2022, 10:35 AM

Thanks for your feedback @ardumont

you're welcome.

But please, do mind it for next time

For sure. Today, I've officially started my work with swh. I'll be proactive now ;)

Great, welcome then.
And thx in advance.

Few things:

  • I have discovered an issue in the Gogs API. Please see T1721#88798

I'll check again later as i've read it but i did not completely got it.

  • There are tons of similarities between Gitea and Gogs.

Can you please describe a bit more what's common?

Should we consider sharing code between the two?

If that's not too much of a work, why not. You can try and propose something in another
diff building on this one if you really want to.

But from what i've read from the runtime code (not the tests yet), the current
implementation is landable as it is.

Cheers!

Same ;)

ardumont accepted this revision.Aug 2 2022, 10:41 AM

lgtm

Thanks!

one remark on the main class docstring that needs a tad rework to match the actual pagination implementation.

swh/lister/gogs/lister.py
37

Please reword the pagination part to match the 'Link>next' header implementation you are now using.

This revision is now accepted and ready to land.Aug 2 2022, 10:41 AM
ardumont mentioned this in T1721: Implementation of Gogs Lister.Aug 2 2022, 12:16 PM
KShivendu updated this revision to Diff 29513.Aug 3 2022, 10:36 AM
  • update GogsLister docstring
  • squash commits
swh-public-ci added a comment.Aug 3 2022, 10:40 AM

Build is green

Patch application report for D8160 (id=29513)

Rebasing onto 1bf11aa26d...

Current branch diff-target is up to date.
Changes applied before test
commit 8d72ef876d23f6929d1748675c961e6dbbd280ee
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 09:35:33 2022 +0530

    gogs: Introduce Gogs lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/569/ for more details.

Harbormaster completed remote builds in B30580: Diff 29513.Aug 3 2022, 10:40 AM
KShivendu updated this revision to Diff 29514.Aug 3 2022, 12:52 PM
KShivendu marked an inline comment as done.
  • minor update to GogsLister docstring
  • squash commit (again)
swh-public-ci added a comment.Aug 3 2022, 12:57 PM

Build is green

Patch application report for D8160 (id=29514)

Rebasing onto 1bf11aa26d...

Current branch diff-target is up to date.
Changes applied before test
commit d34a6232a6934c494033c1fdd71e178b838985ed
Author: KShivendu <shivendu@iitbhilai.ac.in>
Date:   Sat Jul 23 09:35:33 2022 +0530

    gogs: Introduce Gogs lister

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/570/ for more details.

Harbormaster completed remote builds in B30581: Diff 29514.Aug 3 2022, 12:57 PM
Closed by commit rDLSd34a6232a693: gogs: Introduce Gogs lister (authored by KShivendu). · Explain WhyAug 3 2022, 1:08 PM
This revision was automatically updated to reflect the committed changes.
KShivendu added a commit: rDLSd34a6232a693: gogs: Introduce Gogs lister.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
1 line--
1 line--
1 line--
swh/
lister/
gogs/
7 lines100%
Loading...
127 lines96%
Loading...
28 lines100%
Loading...
tests/
81%
Loading...
data/
https_try.gogs.io/
98 lines--
98 lines--
131 lines100%
Loading...
61 lines100%
Loading...
tests/
4 lines100%
Loading...

Diff 29468

View Options

CONTRIBUTORS

Loading...
View Options

README.md

Loading...
View Options

setup.py

Loading...
View Options

swh/lister/gogs/__init__.py

Loading...
View Options

swh/lister/gogs/lister.py

Loading...
View Options

swh/lister/gogs/tasks.py

Loading...
View Options

swh/lister/gogs/tests/__init__.py

Loading...
View Options

swh/lister/gogs/tests/data/https_try.gogs.io/repos_page1

Loading...
View Options

swh/lister/gogs/tests/data/https_try.gogs.io/repos_page2

Loading...
View Options

swh/lister/gogs/tests/test_lister.py

Loading...
View Options

swh/lister/gogs/tests/test_tasks.py

Loading...
View Options

swh/lister/tests/test_cli.py

Loading...
About · Developer information · Legal