This will allow the Git loader to incrementally load GitHub forks
Details
Details
- Reviewers
olasd - Group Reviewers
Reviewers - Maniphest Tasks
- T3273: Use "fork" relationships to speed-up initial load of large repositories
- Commits
- rDLDMD01d34fa76821: Add method get_parent_origins()
Diff Detail
Diff Detail
- Repository
- rDLDMD Extrinsic Metadata Loaders
- Branch
- master
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 28878 Build 45138: Phabricator diff pipeline on jenkins Jenkins console · Jenkins Build 45137: arc lint + arc unit
Event Timeline
Comment Actions
Build is green
Patch application report for D7663 (id=27728)
Rebasing onto aa0af83a5e...
Current branch diff-target is up to date.
Changes applied before test
commit 427d6cdb5f72b9955023b81c6e060f7f2d4cd1c5
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Tue Apr 26 13:28:37 2022 +0200
Add method get_parent_origin()
This will allow the Git loader to incrementally load GitHub forksSee https://jenkins.softwareheritage.org/job/DLDMD/job/tests-on-diff/6/ for more details.
Comment Actions
Should we be using the parent, or the source (which, afaik, is the root of all forks) repo? Or both?
Ideally we'd have a way to attempt all known parent repos in succession, from the closest to the farthest
| swh/loader/metadata/github.py | ||
|---|---|---|
| 79–81 | Maybe we should build that from clone_url (stripping the .git ending if it's there) instead. I guess it's consistent with the way the lister does it? | |
Comment Actions
- use html_url like listers, instead of building the URL
- support both parent and source
Comment Actions
Build is green
Patch application report for D7663 (id=27788)
Rebasing onto aa0af83a5e...
Current branch diff-target is up to date.
Changes applied before test
commit 263515f3538550213159ce1706d30d4e4c015d7c
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date: Tue Apr 26 13:28:37 2022 +0200
Add method get_parent_origin()
This will allow the Git loader to incrementally load GitHub forksSee https://jenkins.softwareheritage.org/job/DLDMD/job/tests-on-diff/7/ for more details.
Comment Actions
Build is green
Patch application report for D7663 (id=27817)
Rebasing onto 01d34fa768...
First, rewinding head to replay your work on top of it... Fast-forwarded diff-target to base-revision-8-D7663.
Changes applied before test
See https://jenkins.softwareheritage.org/job/DLDMD/job/tests-on-diff/8/ for more details.