Page MenuHomeSoftware Heritage
Differential D6286

gitlab: Allow ingestion of hg_git origins as hg ones
ClosedPublic
Actions

Authored by ardumont on Sep 17 2021, 11:51 AM.

Details

Reviewers
anlambert
Group Reviewers
Reviewers
Maniphest Tasks
T3581: List heptapod instance foss.heptapod.net
Commits
rDLSfdb420238cc8: gitlab: Allow ingestion of hg_git origins as hg ones
Summary

Related to T3581#70593

Test Plan

tox

Diff Detail

Repository
rDLS Listers
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Sep 17 2021, 11:51 AM
ardumont updated this revision to Diff 22773.Sep 17 2021, 11:54 AM

Refactor a bit

swh-public-ci added a comment.Sep 17 2021, 11:54 AM

Build is green

Patch application report for D6286 (id=22772)

Rebasing onto 4e4edee478...

Current branch diff-target is up to date.
Changes applied before test
commit 1cf34613cbb903fd288b057c7f86e67ed9af9b00
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Sep 17 11:50:58 2021 +0200

    gitlab: Allow ingestion of hg_git origins as hg ones
    
    Related to T3581#70593

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/352/ for more details.

Harbormaster completed remote builds in B23702: Diff 22772.Sep 17 2021, 11:54 AM
swh-public-ci added a comment.Sep 17 2021, 11:57 AM

Build is green

Patch application report for D6286 (id=22773)

Rebasing onto 4e4edee478...

Current branch diff-target is up to date.
Changes applied before test
commit 2973e841fbc166e7f02da1f553e40a279e54635e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Sep 17 11:50:58 2021 +0200

    gitlab: Allow ingestion of hg_git origins as hg ones
    
    Related to T3581#70593

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/353/ for more details.

Harbormaster completed remote builds in B23703: Diff 22773.Sep 17 2021, 11:57 AM
ardumont requested review of this revision.Sep 17 2021, 11:57 AM
anlambert added a subscriber: anlambert.Sep 17 2021, 12:07 PM
anlambert added inline comments.
swh/lister/gitlab/lister.py
26

I would remove the hg to hg mapping here.

28

not sure if we need this global variable here.

211–212

How about splitting this into two instructions, something like:

visit_type = repo.get("vcs_type", "git")
visit_type = VCS_MAPPING.get(visit_type, visit_type)

This is more readable this way imho.

swh/lister/gitlab/tests/test_lister.py
74–75

I would rather check the visit_type is hg in the listed origins here.

ardumont marked 3 inline comments as done.Sep 17 2021, 12:11 PM
ardumont added inline comments.
swh/lister/gitlab/lister.py
26

you can't otherwise, hg are mapped to git.

>           assert VCS_MAPPING[entry["vcs_type"]] == "hg"
E           KeyError: 'hg'
211–212

ok

swh/lister/gitlab/tests/test_lister.py
74–75

It's both hg_git and hg.
Hency why i added this and the check below becomes clearer.

But i can think of something else then ;)

ardumont updated this revision to Diff 22774.Sep 17 2021, 12:13 PM
ardumont marked 2 inline comments as done.

Adapt according to suggestions to make code clearer! Thanks.

ardumont marked an inline comment as done.Sep 17 2021, 12:14 PM
ardumont added inline comments.
swh/lister/gitlab/lister.py
28

ok, dropped.

swh-public-ci added a comment.Sep 17 2021, 12:16 PM

Build is green

Patch application report for D6286 (id=22774)

Rebasing onto 4e4edee478...

Current branch diff-target is up to date.
Changes applied before test
commit 0c5d9a5306f6adb26a07a188fec6ac66da2d1321
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Sep 17 11:50:58 2021 +0200

    gitlab: Allow ingestion of hg_git origins as hg ones
    
    Related to T3581#70593

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/354/ for more details.

Harbormaster completed remote builds in B23704: Diff 22774.Sep 17 2021, 12:16 PM
ardumont updated this revision to Diff 22775.Sep 17 2021, 12:17 PM

Drop the hg to hg mapping, it's indeed not needed. I must have cross wired my mind and
fingers when testing that.

swh/lister/gitlab/lister.py
26

never mind this, i dropped it indeed.

swh-public-ci added a comment.Sep 17 2021, 12:20 PM

Build is green

Patch application report for D6286 (id=22775)

Rebasing onto 4e4edee478...

Current branch diff-target is up to date.
Changes applied before test
commit fdb420238cc861e6d55e7f8a0ea73b938f83ae47
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Sep 17 11:50:58 2021 +0200

    gitlab: Allow ingestion of hg_git origins as hg ones
    
    Related to T3581#70593

See https://jenkins.softwareheritage.org/job/DLS/job/tests-on-diff/355/ for more details.

Harbormaster completed remote builds in B23705: Diff 22775.Sep 17 2021, 12:20 PM
anlambert added inline comments.Sep 17 2021, 1:12 PM
swh/lister/gitlab/tests/test_lister.py
74–75

I was thinking about that check instead:

scheduler_origins = lister.scheduler.get_listed_origins(
    lister.lister_obj.id
).results
assert all([origin.visit_type == "hg" for origin in scheduler_origins])
ardumont added inline comments.Sep 17 2021, 1:20 PM
swh/lister/gitlab/tests/test_lister.py
74–75

That test is already done below line 86 in the original code though.

Here, i'm merely pointing out that the input does indeed have more vcs-type than usual.
And then that it's mapped to hg (with the existing check).

Isn't it enough?

85

Here is the test already done as mentioned above.

anlambert accepted this revision.Sep 17 2021, 1:26 PM
anlambert added inline comments.
swh/lister/gitlab/tests/test_lister.py
74–75

Ah right, it was hidden by Phabricator UI, I think we are good then.

This revision is now accepted and ready to land.Sep 17 2021, 1:26 PM
Closed by commit rDLSfdb420238cc8: gitlab: Allow ingestion of hg_git origins as hg ones (authored by ardumont). · Explain WhySep 17 2021, 1:37 PM
This revision was automatically updated to reflect the committed changes.
ardumont added a commit: rDLSfdb420238cc8: gitlab: Allow ingestion of hg_git origins as hg ones.
ardumont mentioned this in T3581: List heptapod instance foss.heptapod.net.Sep 17 2021, 2:36 PM

Revision Contents
Changeset List

PathSize
swh/
lister/
gitlab/
12 lines
tests/
9 lines

Diff 22781

View Options

swh/lister/gitlab/lister.py

Loading...
View Options

swh/lister/gitlab/tests/test_lister.py

Loading...
About · Developer information · Legal