Page MenuHomeSoftware Heritage
Paste P128

Mirroring forge from phab to github
ActivePublic
Actions

Authored by ardumont on Jan 27 2017, 12:29 PM.
#+title: Mirroring forge from phab to github
#+author: ardumont, olasd
* Need
- the first step is writing a script that takes the URL (or ID) of
*one* of our repos, and does the entire mirroring setup using the
github API and the phabricator API
- having that script will also make it easy to maintain the sync in
the future when we create new repos
* (forge) API token
generation: https://forge.softwareheritage.org/settings/user/ardumont/page/apitokens/
No setup needed, the token will be passed along to the command.
* (github) Create a public repository
With:
- name/description as in the forge
- wiki disabled
- issues disabled
https://developer.github.com/v3/repos/#create
** API (wip)
curl -H 'Authorization: token <user-token>' -X POST -d @/tmp/tmp.data https://api.github.com/v3/orgs/SoftwareHeritage/repos
/tmp/tmp.data:
#+BEGIN_SRC json
{
"name": "<REPOSITORY>",
"description": "<DESCRIPTION>",
"homepage": "<forge's uri>",
"private": false,
"has_issues": false,
"has_wiki": false,
"has_downloads": true
}
#+END_SRC
Note:
User token is dependent on the user's profile.
When creating the token, it should have 'public_repo'
And the user must be a member of SoftwareHeritage.
* (forge) Edit repository URI
Edit the repository settings on forge.softwareheritage.org:
- Manage repository > URIs (see below for api call)
- Edit the credentials for that new uri to be K2
Values:
- Uri: github's ui
- I/O type: Mirror
- Display type: Hidden
Equivalent api call: https://forge.softwareheritage.org/conduit/method/diffusion.uri.edit/
|------------+---------------------------------------------------------------------------------------------------------------------------------------------------------|
| parameter | Description |
|------------+---------------------------------------------------------------------------------------------------------------------------------------------------------|
| repository | Create a URI in a given repository. This transaction type must be present when creating a new URI and must not be present when editing an existing URI. |
| uri | Change the repository URI. |
| io | Adjust I/O behavior. |
| display | Change display behavior. |
| credential | Change the credential for this URI. |
| disable | Disable or activate the URI. |
|------------+---------------------------------------------------------------------------------------------------------------------------------------------------------|
Note: For a repository: https://forge.softwareheritage.org/conduit/method/diffusion.repository.edit/
A sample call would then be (thank olasd for the adaptation):
#+BEGIN_SRC sh
API_TOKEN="" # token generated in setup phase
REPOSITORY_PHID="" # repository concerned by the mirroring (callsign?)
GITHUB_URI="" # github's mirror repository uri
curl https://forge.softwareheritage.org/api/diffusion.uri.edit \
-d api.token=$API_TOKEN \
-d transactions[0][type]=repository \
-d transactions[0][value]=$REPOSITORY_PHID \
-d transactions[1][type]=uri \
-d transactions[1][value]=$GITHUB_URI \
-d transactions[2][type]=io \
-d transactions[2][value]=mirror \
-d transactions[3][type]=display \
-d transactions[3][value]=never \
-d transactions[4][type]=credential \
-d transactions[4][value]=<K2-PHID>
#+END_SRC
* Test
- [ ] Create a dumb repository in phab's forge
- [ ] Trigger the script for that repository
- [ ] check results on github. Is the repository mirrored?
- [ ] Check results in forge. Is the repository's original uri ok?
- [ ] Push a new commit and push through forge, wait for the magic to happen
- [ ] Is that commit pushed on github's mirror? (expected answer is yes here :)

Event Timeline

ardumont created this paste.Jan 27 2017, 12:29 PM
ardumont created this object with edit policy "Staff (Project)".
ardumont mentioned this in T619: maintain an up-to-date mirror of forge.softwareheritage.org code on GitHub.
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 1:06 PM
olasd added a subscriber: olasd.Jan 27 2017, 2:46 PM

Here's a proper call for diffusion.uri.edit:

curl https://forge.softwareheritage.org/api/diffusion.uri.edit \
-d api.token=api-token \
-d transactions[0][type]=repository \
-d transactions[0][value]=repositoryPHID \
-d transactions[1][type]=uri \
-d transactions[1][value]=git%40github.com%3Asoftwareheritage/repo.git \
-d transactions[2][type]=io \
-d transactions[2][value]=mirror \
-d transactions[3][type]=display \
-d transactions[3][value]=never \
-d transactions[4][type]=credential \
-d transactions[4][value]=PHID-CDTL-keoenkcfmbq2nn3dgor6
olasd added a comment.Jan 27 2017, 3:01 PM

Listing all publically accessible repositories:

curl https://forge.softwareheritage.org/api/diffusion.repository.search \
  -d api.token=api-token \
  -d queryKey=all | jq '.result.data | map(select(.fields.policy.view == "public"))'
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 3:05 PM
olasd added a comment.Jan 27 2017, 3:16 PM

Listing all repos tagged with the Sync to GitHub tag:

curl https://forge.softwareheritage.org/api/diffusion.repository.search \
  -d api.token=api-token \
  -d constraints[projects][0]=PHID-PROJ-ximqdentnggqkdbc2ycg
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 3:18 PM
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 3:24 PM
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 3:38 PM
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 3:41 PM
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 3:55 PM
ardumont edited the content of this paste. (Show Details)
ardumont edited the content of this paste. (Show Details)Jan 27 2017, 4:41 PM
ardumont mentioned this in rDSNIP86b2963e4b98: WIP - Add automation script for repository mirroring from the forge to github.Jan 27 2017, 6:15 PM
About · Developer information · Legal