Page MenuHomeSoftware Heritage
Create Task
Maniphest T4548

Add a public API endpoint and documentation to trigger Save Code Now from webhook
Closed, MigratedEdits Locked
Actions

Description

Idea from @rdicosmo:

As an alternative to the "save to SWH" github action, we could provide an API endpoint that can be called by forges (github, gitlab, gitea/gogs, ...) to notify us of a push or new tag.

Motivation:

  1. less wasteful on the forge's side (it's just an HTTPS request, not a container with its own nodejs process, ...)
  2. easier to setup for users (a few clicks in the forge's UI)
  3. more secure (no need to trust the action's author)
  4. works for all forges
  5. might allow us to extract information from the webhook payload, in the future
  6. ...

Revisions and Commits

rDWAPPS Web applications
D8902rDWAPPSf392e87cf449 save_code_now: Indicate if request was created from webhook in Web UI
D8890rDWAPPSb4fc49b3fe92 metrics: Add gauge counting save origin requests created from webhooks
D8890rDWAPPSce8da30b9842 metrics/tests/test_app: Fix test name
D8889rDWAPPS12a291957fa0 save_code_now: Add webhook info fields to SaveOriginRequest model
D8868rDWAPPS1b8992d585a0 save_origin_webhooks: Handle private repositories
D8867rDWAPPS84922dfd698b save_origin_webhooks: Handle github ping event
D8841rDWAPPSab849109ea1d save_code_now: Allow request creation if previous for origin is running
D8798rDWAPPS93696b9cc7b9 save_origin_webhooks: Add push webhook receivers for popular forges

Related Objects

Event Timeline

vlorentz triaged this task as Normal priority.Sep 22 2022, 2:51 PM
vlorentz created this task.
vlorentz added a project: Web app.
anlambert added a subscriber: anlambert.Oct 25 2022, 4:22 PM

I started looking how to implement that task.

We could add the following endpoint to the Web API: /api/1/origin/save/webhook/ accepting POST requests only with content type application/json.

We can add support for the following forge webhooks:

Based on the received HTTP POST request headers, we should be able to determine which type of forge sent us the request and then
extract repository URL and visit type from the JSON payload in order to create a Save Code Now request when new commits are pushed.

anlambert added a comment.Oct 25 2022, 5:21 PM

I used ngrok to forward webhook requests to my local machine.

This is the POST request I received from GitHub when pushing a new commit to a sample repository.

Headers
Request URL: https://c5d9-128-93-73-169.eu.ngrok.io/api/1/origin/save/webhook/
Request method: POST
Accept: */*
content-type: application/json
User-Agent: GitHub-Hookshot/ede37db
X-GitHub-Delivery: 6b5bb884-5477-11ed-8d56-770b99aba934
X-GitHub-Event: push
X-GitHub-Hook-ID: 385510251
X-GitHub-Hook-Installation-Target-ID: 557370172
X-GitHub-Hook-Installation-Target-Type: repository
Payload
{
  "ref": "refs/heads/main",
  "before": "63c72ea620a8d93aa14f0d60b9320881631c83ca",
  "after": "366675b3a4017eb3ace4452d8288ae664c4f0af4",
  "repository": {
    "id": 557370172,
    "node_id": "R_kgDOITjLPA",
    "name": "webhook-test",
    "full_name": "anlambert/webhook-test",
    "private": false,
    "owner": {
      "name": "anlambert",
      "email": "antoine.lambert33@gmail.com",
      "login": "anlambert",
      "id": 5493543,
      "node_id": "MDQ6VXNlcjU0OTM1NDM=",
      "avatar_url": "https://avatars.githubusercontent.com/u/5493543?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/anlambert",
      "html_url": "https://github.com/anlambert",
      "followers_url": "https://api.github.com/users/anlambert/followers",
      "following_url": "https://api.github.com/users/anlambert/following{/other_user}",
      "gists_url": "https://api.github.com/users/anlambert/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/anlambert/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/anlambert/subscriptions",
      "organizations_url": "https://api.github.com/users/anlambert/orgs",
      "repos_url": "https://api.github.com/users/anlambert/repos",
      "events_url": "https://api.github.com/users/anlambert/events{/privacy}",
      "received_events_url": "https://api.github.com/users/anlambert/received_events",
      "type": "User",
      "site_admin": false
    },
    "html_url": "https://github.com/anlambert/webhook-test",
    "description": "Sample repository to test webhooks delivery",
    "fork": false,
    "url": "https://github.com/anlambert/webhook-test",
    "forks_url": "https://api.github.com/repos/anlambert/webhook-test/forks",
    "keys_url": "https://api.github.com/repos/anlambert/webhook-test/keys{/key_id}",
    "collaborators_url": "https://api.github.com/repos/anlambert/webhook-test/collaborators{/collaborator}",
    "teams_url": "https://api.github.com/repos/anlambert/webhook-test/teams",
    "hooks_url": "https://api.github.com/repos/anlambert/webhook-test/hooks",
    "issue_events_url": "https://api.github.com/repos/anlambert/webhook-test/issues/events{/number}",
    "events_url": "https://api.github.com/repos/anlambert/webhook-test/events",
    "assignees_url": "https://api.github.com/repos/anlambert/webhook-test/assignees{/user}",
    "branches_url": "https://api.github.com/repos/anlambert/webhook-test/branches{/branch}",
    "tags_url": "https://api.github.com/repos/anlambert/webhook-test/tags",
    "blobs_url": "https://api.github.com/repos/anlambert/webhook-test/git/blobs{/sha}",
    "git_tags_url": "https://api.github.com/repos/anlambert/webhook-test/git/tags{/sha}",
    "git_refs_url": "https://api.github.com/repos/anlambert/webhook-test/git/refs{/sha}",
    "trees_url": "https://api.github.com/repos/anlambert/webhook-test/git/trees{/sha}",
    "statuses_url": "https://api.github.com/repos/anlambert/webhook-test/statuses/{sha}",
    "languages_url": "https://api.github.com/repos/anlambert/webhook-test/languages",
    "stargazers_url": "https://api.github.com/repos/anlambert/webhook-test/stargazers",
    "contributors_url": "https://api.github.com/repos/anlambert/webhook-test/contributors",
    "subscribers_url": "https://api.github.com/repos/anlambert/webhook-test/subscribers",
    "subscription_url": "https://api.github.com/repos/anlambert/webhook-test/subscription",
    "commits_url": "https://api.github.com/repos/anlambert/webhook-test/commits{/sha}",
    "git_commits_url": "https://api.github.com/repos/anlambert/webhook-test/git/commits{/sha}",
    "comments_url": "https://api.github.com/repos/anlambert/webhook-test/comments{/number}",
    "issue_comment_url": "https://api.github.com/repos/anlambert/webhook-test/issues/comments{/number}",
    "contents_url": "https://api.github.com/repos/anlambert/webhook-test/contents/{+path}",
    "compare_url": "https://api.github.com/repos/anlambert/webhook-test/compare/{base}...{head}",
    "merges_url": "https://api.github.com/repos/anlambert/webhook-test/merges",
    "archive_url": "https://api.github.com/repos/anlambert/webhook-test/{archive_format}{/ref}",
    "downloads_url": "https://api.github.com/repos/anlambert/webhook-test/downloads",
    "issues_url": "https://api.github.com/repos/anlambert/webhook-test/issues{/number}",
    "pulls_url": "https://api.github.com/repos/anlambert/webhook-test/pulls{/number}",
    "milestones_url": "https://api.github.com/repos/anlambert/webhook-test/milestones{/number}",
    "notifications_url": "https://api.github.com/repos/anlambert/webhook-test/notifications{?since,all,participating}",
    "labels_url": "https://api.github.com/repos/anlambert/webhook-test/labels{/name}",
    "releases_url": "https://api.github.com/repos/anlambert/webhook-test/releases{/id}",
    "deployments_url": "https://api.github.com/repos/anlambert/webhook-test/deployments",
    "created_at": 1666710385,
    "updated_at": "2022-10-25T15:06:25Z",
    "pushed_at": 1666710739,
    "git_url": "git://github.com/anlambert/webhook-test.git",
    "ssh_url": "git@github.com:anlambert/webhook-test.git",
    "clone_url": "https://github.com/anlambert/webhook-test.git",
    "svn_url": "https://github.com/anlambert/webhook-test",
    "homepage": null,
    "size": 0,
    "stargazers_count": 0,
    "watchers_count": 0,
    "language": null,
    "has_issues": true,
    "has_projects": true,
    "has_downloads": true,
    "has_wiki": true,
    "has_pages": false,
    "forks_count": 0,
    "mirror_url": null,
    "archived": false,
    "disabled": false,
    "open_issues_count": 0,
    "license": null,
    "allow_forking": true,
    "is_template": false,
    "web_commit_signoff_required": false,
    "topics": [

    ],
    "visibility": "public",
    "forks": 0,
    "open_issues": 0,
    "watchers": 0,
    "default_branch": "main",
    "stargazers": 0,
    "master_branch": "main"
  },
  "pusher": {
    "name": "anlambert",
    "email": "antoine.lambert33@gmail.com"
  },
  "sender": {
    "login": "anlambert",
    "id": 5493543,
    "node_id": "MDQ6VXNlcjU0OTM1NDM=",
    "avatar_url": "https://avatars.githubusercontent.com/u/5493543?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/anlambert",
    "html_url": "https://github.com/anlambert",
    "followers_url": "https://api.github.com/users/anlambert/followers",
    "following_url": "https://api.github.com/users/anlambert/following{/other_user}",
    "gists_url": "https://api.github.com/users/anlambert/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/anlambert/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/anlambert/subscriptions",
    "organizations_url": "https://api.github.com/users/anlambert/orgs",
    "repos_url": "https://api.github.com/users/anlambert/repos",
    "events_url": "https://api.github.com/users/anlambert/events{/privacy}",
    "received_events_url": "https://api.github.com/users/anlambert/received_events",
    "type": "User",
    "site_admin": false
  },
  "created": false,
  "deleted": false,
  "forced": false,
  "base_ref": null,
  "compare": "https://github.com/anlambert/webhook-test/compare/63c72ea620a8...366675b3a401",
  "commits": [
    {
      "id": "366675b3a4017eb3ace4452d8288ae664c4f0af4",
      "tree_id": "e824f22f6528dfae5489e6a08888f460eb74f975",
      "distinct": true,
      "message": "Add a new line to trigger webhook when pushing commit",
      "timestamp": "2022-10-25T17:12:09+02:00",
      "url": "https://github.com/anlambert/webhook-test/commit/366675b3a4017eb3ace4452d8288ae664c4f0af4",
      "author": {
        "name": "Antoine Lambert",
        "email": "anlambert@softwareheritage.org",
        "username": "anlambert"
      },
      "committer": {
        "name": "Antoine Lambert",
        "email": "anlambert@softwareheritage.org",
        "username": "anlambert"
      },
      "added": [

      ],
      "removed": [

      ],
      "modified": [
        "README.md"
      ]
    }
  ],
  "head_commit": {
    "id": "366675b3a4017eb3ace4452d8288ae664c4f0af4",
    "tree_id": "e824f22f6528dfae5489e6a08888f460eb74f975",
    "distinct": true,
    "message": "Add a new line to trigger webhook when pushing commit",
    "timestamp": "2022-10-25T17:12:09+02:00",
    "url": "https://github.com/anlambert/webhook-test/commit/366675b3a4017eb3ace4452d8288ae664c4f0af4",
    "author": {
      "name": "Antoine Lambert",
      "email": "anlambert@softwareheritage.org",
      "username": "anlambert"
    },
    "committer": {
      "name": "Antoine Lambert",
      "email": "anlambert@softwareheritage.org",
      "username": "anlambert"
    },
    "added": [

    ],
    "removed": [

    ],
    "modified": [
      "README.md"
    ]
  }
}
anlambert added a comment.Oct 25 2022, 5:27 PM

This is the POST request I received from GitLab when pushing commit to a sample repository.

Headers
Content-Type: application/json
User-Agent: GitLab/15.6.0-pre
X-Gitlab-Event: Push Hook
X-Gitlab-Instance: https://gitlab.com
X-Gitlab-Event-UUID: 02dd0cbd-bf4b-4774-ad24-8c7752972aae
Payload
{
  "object_kind": "push",
  "event_name": "push",
  "before": "1b0392a66f1439bac26c0845a083fe64f01a00fe",
  "after": "87b5530bd21f10330bc1437296295205112088c5",
  "ref": "refs/heads/main",
  "checkout_sha": "87b5530bd21f10330bc1437296295205112088c5",
  "message": null,
  "user_id": 4775591,
  "user_name": "Antoine Lambert",
  "user_username": "anlambert",
  "user_email": "",
  "user_avatar": "https://secure.gravatar.com/avatar/1c41d79af82e53f20334d1cf9a3dea7e?s=80&d=identicon",
  "project_id": 39639957,
  "project": {
    "id": 39639957,
    "name": "webhook-test",
    "description": "",
    "web_url": "https://gitlab.com/anlambert/test",
    "avatar_url": null,
    "git_ssh_url": "git@gitlab.com:anlambert/test.git",
    "git_http_url": "https://gitlab.com/anlambert/test.git",
    "namespace": "Antoine Lambert",
    "visibility_level": 20,
    "path_with_namespace": "anlambert/test",
    "default_branch": "main",
    "ci_config_path": "",
    "homepage": "https://gitlab.com/anlambert/test",
    "url": "git@gitlab.com:anlambert/test.git",
    "ssh_url": "git@gitlab.com:anlambert/test.git",
    "http_url": "https://gitlab.com/anlambert/test.git"
  },
  "commits": [
    {
      "id": "87b5530bd21f10330bc1437296295205112088c5",
      "message": "Remove line to trigger webhook when pushing commit\n",
      "title": "Remove line to trigger webhook when pushing commit",
      "timestamp": "2022-10-25T17:24:50+02:00",
      "url": "https://gitlab.com/anlambert/test/-/commit/87b5530bd21f10330bc1437296295205112088c5",
      "author": {
        "name": "Antoine Lambert",
        "email": "[REDACTED]"
      },
      "added": [

      ],
      "modified": [
        "README.md"
      ],
      "removed": [

      ]
    }
  ],
  "total_commits_count": 1,
  "push_options": {
  },
  "repository": {
    "name": "webhook-test",
    "url": "git@gitlab.com:anlambert/test.git",
    "description": "",
    "homepage": "https://gitlab.com/anlambert/test",
    "git_http_url": "https://gitlab.com/anlambert/test.git",
    "git_ssh_url": "git@gitlab.com:anlambert/test.git",
    "visibility_level": 20
  }
}
anlambert added a comment.Oct 25 2022, 5:36 PM

This is the POST request I received from Gitea when pushing commit to a sample repository.

Headers
Request URL: https://c5d9-128-93-73-169.eu.ngrok.io/api/1/origin/save/webhook/
Request method: POST
Content-Type: application/json
X-GitHub-Delivery: 9a1c1cd7-d515-4ef5-a699-01aadf500270
X-GitHub-Event: push
X-GitHub-Event-Type: push
X-Gitea-Delivery: 9a1c1cd7-d515-4ef5-a699-01aadf500270
X-Gitea-Event: push
X-Gitea-Event-Type: push
X-Gitea-Signature: 
X-Gogs-Delivery: 9a1c1cd7-d515-4ef5-a699-01aadf500270
X-Gogs-Event: push
X-Gogs-Event-Type: push
X-Gogs-Signature: 
X-Hub-Signature: sha1=
X-Hub-Signature-256: sha256=
Payload
{
  "ref": "refs/heads/main",
  "before": "d74a0d9989b1642e82e5f7ab3ba395f4034b1fcd",
  "after": "fc0cf34aac1443223c9dd55f3a901e6b38bd25b0",
  "compare_url": "https://try.gitea.io/anlambert/webhook-test/compare/d74a0d9989b1642e82e5f7ab3ba395f4034b1fcd...fc0cf34aac1443223c9dd55f3a901e6b38bd25b0",
  "commits": [
    {
      "id": "fc0cf34aac1443223c9dd55f3a901e6b38bd25b0",
      "message": "Add a new line to trigger webhook when pushing commit\n",
      "url": "https://try.gitea.io/anlambert/webhook-test/commit/fc0cf34aac1443223c9dd55f3a901e6b38bd25b0",
      "author": {
        "name": "Antoine Lambert",
        "email": "anlambert@softwareheritage.org",
        "username": ""
      },
      "committer": {
        "name": "Antoine Lambert",
        "email": "anlambert@softwareheritage.org",
        "username": ""
      },
      "verification": null,
      "timestamp": "2022-10-25T17:34:08+02:00",
      "added": [],
      "removed": [],
      "modified": [
        "README.md"
      ]
    }
  ],
  "total_commits": 1,
  "head_commit": {
    "id": "fc0cf34aac1443223c9dd55f3a901e6b38bd25b0",
    "message": "Add a new line to trigger webhook when pushing commit\n",
    "url": "https://try.gitea.io/anlambert/webhook-test/commit/fc0cf34aac1443223c9dd55f3a901e6b38bd25b0",
    "author": {
      "name": "Antoine Lambert",
      "email": "anlambert@softwareheritage.org",
      "username": ""
    },
    "committer": {
      "name": "Antoine Lambert",
      "email": "anlambert@softwareheritage.org",
      "username": ""
    },
    "verification": null,
    "timestamp": "2022-10-25T17:34:08+02:00",
    "added": [],
    "removed": [],
    "modified": [
      "README.md"
    ]
  },
  "repository": {
    "id": 38171,
    "owner": {
      "id": 520163,
      "login": "anlambert",
      "login_name": "",
      "full_name": "",
      "email": "anlambert@noreply.try.gitea.io",
      "avatar_url": "https://try.gitea.io/avatars/1c41d79af82e53f20334d1cf9a3dea7e",
      "language": "",
      "is_admin": false,
      "last_login": "0001-01-01T00:00:00Z",
      "created": "2022-10-25T15:30:24Z",
      "restricted": false,
      "active": false,
      "prohibit_login": false,
      "location": "",
      "website": "",
      "description": "",
      "visibility": "public",
      "followers_count": 0,
      "following_count": 0,
      "starred_repos_count": 0,
      "username": "anlambert"
    },
    "name": "webhook-test",
    "full_name": "anlambert/webhook-test",
    "description": "",
    "empty": false,
    "private": false,
    "fork": false,
    "template": false,
    "parent": null,
    "mirror": false,
    "size": 88,
    "language": "",
    "languages_url": "https://try.gitea.io/api/v1/repos/anlambert/webhook-test/languages",
    "html_url": "https://try.gitea.io/anlambert/webhook-test",
    "ssh_url": "git@try.gitea.io:anlambert/webhook-test.git",
    "clone_url": "https://try.gitea.io/anlambert/webhook-test.git",
    "original_url": "",
    "website": "",
    "stars_count": 0,
    "forks_count": 0,
    "watchers_count": 1,
    "open_issues_count": 0,
    "open_pr_counter": 0,
    "release_counter": 0,
    "default_branch": "main",
    "archived": false,
    "created_at": "2022-10-25T15:31:27Z",
    "updated_at": "2022-10-25T15:31:28Z",
    "permissions": {
      "admin": true,
      "push": true,
      "pull": true
    },
    "has_issues": true,
    "internal_tracker": {
      "enable_time_tracker": false,
      "allow_only_contributors_to_track_time": true,
      "enable_issue_dependencies": true
    },
    "has_wiki": true,
    "has_pull_requests": true,
    "has_projects": true,
    "ignore_whitespace_conflicts": false,
    "allow_merge_commits": true,
    "allow_rebase": true,
    "allow_rebase_explicit": true,
    "allow_squash_merge": true,
    "allow_rebase_update": true,
    "default_delete_branch_after_merge": false,
    "default_merge_style": "merge",
    "avatar_url": "",
    "internal": false,
    "mirror_interval": "",
    "mirror_updated": "0001-01-01T00:00:00Z",
    "repo_transfer": null
  },
  "pusher": {
    "id": 520163,
    "login": "anlambert",
    "login_name": "",
    "full_name": "",
    "email": "anlambert@noreply.try.gitea.io",
    "avatar_url": "https://try.gitea.io/avatars/1c41d79af82e53f20334d1cf9a3dea7e",
    "language": "",
    "is_admin": false,
    "last_login": "0001-01-01T00:00:00Z",
    "created": "2022-10-25T15:30:24Z",
    "restricted": false,
    "active": false,
    "prohibit_login": false,
    "location": "",
    "website": "",
    "description": "",
    "visibility": "public",
    "followers_count": 0,
    "following_count": 0,
    "starred_repos_count": 0,
    "username": "anlambert"
  },
  "sender": {
    "id": 520163,
    "login": "anlambert",
    "login_name": "",
    "full_name": "",
    "email": "anlambert@noreply.try.gitea.io",
    "avatar_url": "https://try.gitea.io/avatars/1c41d79af82e53f20334d1cf9a3dea7e",
    "language": "",
    "is_admin": false,
    "last_login": "0001-01-01T00:00:00Z",
    "created": "2022-10-25T15:30:24Z",
    "restricted": false,
    "active": false,
    "prohibit_login": false,
    "location": "",
    "website": "",
    "description": "",
    "visibility": "public",
    "followers_count": 0,
    "following_count": 0,
    "starred_repos_count": 0,
    "username": "anlambert"
  }
}
anlambert added a comment.Oct 25 2022, 5:49 PM

This is the POST request I received from SourceForge when pushing commit to a sample git repository.

Headers
Accept: */*
Accept-Encoding: gzip, deflate
Content-Length: 754
Content-Type: application/json
Host: c5d9-128-93-73-169.eu.ngrok.io
User-Agent: Allura Webhook (https://allura.apache.org/)
X-Allura-Signature: sha1=4e77ddb83a1e14d406191e0b8eab20cc020415b2
X-Forwarded-For: 216.105.38.6
X-Forwarded-Proto: https
Payload
{
    "size": 1,
    "commits": [
        {
            "id": "42157ad6b5d5dee256ba450c9d7a1e70a31bb701",
            "url": "https://sourceforge.net/p/webhook-test-git/code/ci/42157ad6b5d5dee256ba450c9d7a1e70a31bb701/",
            "timestamp": "2022-10-25T15:43:40Z",
            "message": "Add REAME.md",
            "author": {
                "name": "Antoine Lambert",
                "email": "anlambert@softwareheritage.org",
                "username": ""
            },
            "committer": {
                "name": "Antoine Lambert",
                "email": "anlambert@softwareheritage.org",
                "username": ""
            },
            "added": [
                "README.md"
            ],
            "removed": [],
            "modified": [],
            "copied": [],
            "renamed": []
        }
    ],
    "before": "",
    "after": "42157ad6b5d5dee256ba450c9d7a1e70a31bb701",
    "repository": {
        "name": "Code",
        "full_name": "/p/webhook-test-git/code/",
        "url": "https://sourceforge.net/p/webhook-test-git/code/"
    },
    "ref": "refs/heads/master"
}
anlambert added a comment.Oct 25 2022, 5:56 PM

This is the POST request I received from SourceForge when adding commit to a sample svn repository.

Headers
Accept: */*
Accept-Encoding: gzip, deflate
Content-Length: 566
Content-Type: application/json
Host: c5d9-128-93-73-169.eu.ngrok.io
User-Agent: Allura Webhook (https://allura.apache.org/)
X-Allura-Signature: sha1=b83598f4ec4060dddaa749fc285e308b54a76ea8
X-Forwarded-For: 216.105.38.6
X-Forwarded-Proto: https
Payload
{
    "size": 1,
    "commits": [
        {
            "id": "r1",
            "url": "https://sourceforge.net/p/webhook-test-svn/code/1/",
            "timestamp": "2022-10-25T15:53:04Z",
            "message": "Add initial directories",
            "author": {
                "name": "anlambert",
                "email": "",
                "username": ""
            },
            "committer": {
                "name": "anlambert",
                "email": "",
                "username": ""
            },
            "added": [
                "/branches",
                "/tags",
                "/trunk"
            ],
            "removed": [],
            "modified": [],
            "copied": [],
            "renamed": []
        }
    ],
    "before": "",
    "after": "r1",
    "repository": {
        "name": "Code",
        "full_name": "/p/webhook-test-svn/code/",
        "url": "https://sourceforge.net/p/webhook-test-svn/code/"
    }
}
anlambert added a comment.Oct 25 2022, 6:02 PM

This is the POST request I received from SourceForge when pushing commit to a sample hg repository.

Headers
Accept: */*
Accept-Encoding: gzip, deflate
Content-Length: 739
Content-Type: application/json
Host: c5d9-128-93-73-169.eu.ngrok.io
User-Agent: Allura Webhook (https://allura.apache.org/)
X-Allura-Signature: sha1=943374ed55185ee54bd554eb8af110c3981d7583
X-Forwarded-For: 216.105.38.6
X-Forwarded-Proto: https
Payload
{
    "size": 1,
    "commits": [
        {
            "id": "c4e59883381e38e79e9c0418e759392da565629d",
            "url": "https://sourceforge.net/p/webhook-test-hg/code/ci/c4e59883381e38e79e9c0418e759392da565629d/",
            "timestamp": "2022-10-25T15:59:02Z",
            "message": "Initial commit",
            "author": {
                "name": "anlambert",
                "email": "anlambert@softwareheritage.org",
                "username": ""
            },
            "committer": {
                "name": "anlambert",
                "email": "anlambert@softwareheritage.org",
                "username": ""
            },
            "added": [
                "README"
            ],
            "removed": [],
            "modified": [],
            "copied": [],
            "renamed": []
        }
    ],
    "before": "",
    "after": "c4e59883381e38e79e9c0418e759392da565629d",
    "repository": {
        "name": "Code",
        "full_name": "/p/webhook-test-hg/code/",
        "url": "https://sourceforge.net/p/webhook-test-hg/code/"
    },
    "ref": "refs/heads/default"
}
vlorentz added a comment.Oct 26 2022, 11:04 AM
In T4548#97818, @anlambert wrote:

Based on the received HTTP POST request headers, we should be able to determine which type of forge sent us the request and then
extract repository URL and visit type from the JSON payload in order to create a Save Code Now request when new commits are pushed.

We can do that and it certainly works, but once we start allowing a single endpoint for all forges, we cannot go back.

What about a separate path for each forge type, so we have more flexibility later on? eg. to route/"cache" requests from some forges differently directly in Varnish

anlambert added a comment.EditedOct 26 2022, 11:44 AM
In T4548#97884, @vlorentz wrote:
In T4548#97818, @anlambert wrote:

Based on the received HTTP POST request headers, we should be able to determine which type of forge sent us the request and then
extract repository URL and visit type from the JSON payload in order to create a Save Code Now request when new commits are pushed.

We can do that and it certainly works, but once we start allowing a single endpoint for all forges, we cannot go back.

What about a separate path for each forge type, so we have more flexibility later on? eg. to route/"cache" requests from some forges differently directly in Varnish

That's a good idea in terms of flexibility indeed, plus it will avoid to have some kind of spaghetti code to determine forge type if we use a single endpoint.

anlambert added a comment.Oct 26 2022, 11:54 AM

Regarding JSON payload sent by SourceForge, we cannot guess the visit type (git, svn, or hg) from it so we will have to query the project endpoint from SourceForge REST API to get it.

curl https://sourceforge.net/rest/p/webhook-test-svn
{
  "shortname": "webhook-test-svn",
  "name": "webhook-test-svn",
  "_id": "635805ebe3c1ec1e7d0eab36",
  "url": "https://sourceforge.net/p/webhook-test-svn/",
  "private": false,
  "short_description": "",
  "creation_date": "2022-10-25",
  "summary": "",
  "external_homepage": "https://webhook-test-svn.sourceforge.io",
  "video_url": "",
  "socialnetworks": [],
  "status": "active",
  "moved_to_url": "",
  "preferred_support_tool": "",
  "preferred_support_url": "",
  "developers": [
    {
      "username": "anlambert",
      "name": "Antoine Lambert ",
      "url": "https://sourceforge.net/u/anlambert/"
    }
  ],
  "tools": [
    {
      "name": "activity",
      "mount_point": "activity",
      "url": "https://sourceforge.net/p/webhook-test-svn/activity/",
      "mount_label": "Activity",
      "api_url": "https://sourceforge.net/rest/p/webhook-test-svn/activity/"
    },
    {
      "name": "summary",
      "mount_point": "summary",
      "url": "https://sourceforge.net/p/webhook-test-svn/summary/",
      "mount_label": "Summary",
      "sourceforge_group_id": 3568124
    },
    {
      "name": "files-sf",
      "mount_point": "files-sf",
      "url": "https://sourceforge.net/p/webhook-test-svn/files-sf/",
      "mount_label": "Files"
    },
    {
      "name": "reviews",
      "mount_point": "reviews",
      "url": "https://sourceforge.net/p/webhook-test-svn/reviews/",
      "mount_label": "Reviews"
    },
    {
      "name": "support",
      "mount_point": "support",
      "url": "https://sourceforge.net/p/webhook-test-svn/support/",
      "mount_label": "Support"
    },
    {
      "name": "svn",
      "mount_point": "code",
      "url": "https://sourceforge.net/p/webhook-test-svn/code/",
      "mount_label": "Code",
      "api_url": "https://sourceforge.net/rest/p/webhook-test-svn/code/",
      "clone_url_https_anon": "https://svn.code.sf.net/p/webhook-test-svn/code/",
      "clone_url_ro": "svn://svn.code.sf.net/p/webhook-test-svn/code/"
    }
  ],
  "labels": [],
  "categories": {
    "audience": [],
    "developmentstatus": [],
    "environment": [],
    "language": [],
    "license": [],
    "translation": [],
    "os": [],
    "database": [],
    "topic": []
  },
  "icon_url": null,
  "screenshots": []
}
curl https://sourceforge.net/rest/p/webhook-test-git
{
  "shortname": "webhook-test-git",
  "name": "webhook-test-git",
  "_id": "635802e25fe6a2c790df6731",
  "url": "https://sourceforge.net/p/webhook-test-git/",
  "private": false,
  "short_description": "",
  "creation_date": "2022-10-25",
  "summary": "",
  "external_homepage": "https://webhook-test-git.sourceforge.io",
  "video_url": "",
  "socialnetworks": [],
  "status": "active",
  "moved_to_url": "",
  "preferred_support_tool": "",
  "preferred_support_url": "",
  "developers": [
    {
      "username": "anlambert",
      "name": "Antoine Lambert ",
      "url": "https://sourceforge.net/u/anlambert/"
    }
  ],
  "tools": [
    {
      "name": "activity",
      "mount_point": "activity",
      "url": "https://sourceforge.net/p/webhook-test-git/activity/",
      "mount_label": "Activity",
      "api_url": "https://sourceforge.net/rest/p/webhook-test-git/activity/"
    },
    {
      "name": "summary",
      "mount_point": "summary",
      "url": "https://sourceforge.net/p/webhook-test-git/summary/",
      "mount_label": "Summary",
      "sourceforge_group_id": 3568118
    },
    {
      "name": "files-sf",
      "mount_point": "files-sf",
      "url": "https://sourceforge.net/p/webhook-test-git/files-sf/",
      "mount_label": "Files"
    },
    {
      "name": "reviews",
      "mount_point": "reviews",
      "url": "https://sourceforge.net/p/webhook-test-git/reviews/",
      "mount_label": "Reviews"
    },
    {
      "name": "support",
      "mount_point": "support",
      "url": "https://sourceforge.net/p/webhook-test-git/support/",
      "mount_label": "Support"
    },
    {
      "name": "git",
      "mount_point": "code",
      "url": "https://sourceforge.net/p/webhook-test-git/code/",
      "mount_label": "Code",
      "api_url": "https://sourceforge.net/rest/p/webhook-test-git/code/",
      "clone_url_https_anon": "https://git.code.sf.net/p/webhook-test-git/code",
      "clone_url_ro": "git://git.code.sf.net/p/webhook-test-git/code"
    }
  ],
  "labels": [],
  "categories": {
    "audience": [],
    "developmentstatus": [],
    "environment": [],
    "language": [],
    "license": [],
    "translation": [],
    "os": [],
    "database": [],
    "topic": []
  },
  "icon_url": null,
  "screenshots": []
}
curl https://sourceforge.net/rest/p/webhook-test-hg
{
  "shortname": "webhook-test-hg",
  "name": "webhook-test-hg",
  "_id": "63580743079e46f00071122b",
  "url": "https://sourceforge.net/p/webhook-test-hg/",
  "private": false,
  "short_description": "",
  "creation_date": "2022-10-25",
  "summary": "",
  "external_homepage": "https://webhook-test-hg.sourceforge.io",
  "video_url": "",
  "socialnetworks": [],
  "status": "active",
  "moved_to_url": "",
  "preferred_support_tool": "",
  "preferred_support_url": "",
  "developers": [
    {
      "username": "anlambert",
      "name": "Antoine Lambert ",
      "url": "https://sourceforge.net/u/anlambert/"
    }
  ],
  "tools": [
    {
      "name": "activity",
      "mount_point": "activity",
      "url": "https://sourceforge.net/p/webhook-test-hg/activity/",
      "mount_label": "Activity",
      "api_url": "https://sourceforge.net/rest/p/webhook-test-hg/activity/"
    },
    {
      "name": "summary",
      "mount_point": "summary",
      "url": "https://sourceforge.net/p/webhook-test-hg/summary/",
      "mount_label": "Summary",
      "sourceforge_group_id": 3568128
    },
    {
      "name": "files-sf",
      "mount_point": "files-sf",
      "url": "https://sourceforge.net/p/webhook-test-hg/files-sf/",
      "mount_label": "Files"
    },
    {
      "name": "reviews",
      "mount_point": "reviews",
      "url": "https://sourceforge.net/p/webhook-test-hg/reviews/",
      "mount_label": "Reviews"
    },
    {
      "name": "support",
      "mount_point": "support",
      "url": "https://sourceforge.net/p/webhook-test-hg/support/",
      "mount_label": "Support"
    },
    {
      "name": "hg",
      "mount_point": "code",
      "url": "https://sourceforge.net/p/webhook-test-hg/code/",
      "mount_label": "Code",
      "api_url": "https://sourceforge.net/rest/p/webhook-test-hg/code/",
      "clone_url_ro": "http://hg.code.sf.net/p/webhook-test-hg/code"
    }
  ],
  "labels": [],
  "categories": {
    "audience": [],
    "developmentstatus": [],
    "environment": [],
    "language": [],
    "license": [],
    "translation": [],
    "os": [],
    "database": [],
    "topic": []
  },
  "icon_url": null,
  "screenshots": []
}
anlambert added a comment.Oct 26 2022, 12:07 PM

This is the POST request I received from Bitbucket when pushing commit to a sample repository.

Headers
Accept: */*
Accept-Encoding: gzip
Content-Length: 8910
Content-Type: application/json
Host: c5d9-128-93-73-169.eu.ngrok.io
User-Agent: Bitbucket-Webhooks/2.0
X-Attempt-Number: 1
X-B3-Sampled: 1
X-B3-Spanid: 82c3a26fbde83e36
X-B3-Traceid: 82c3a26fbde83e36
X-Event-Key: repo:push
X-Event-Time: Wed, 26 Oct 2022 10:03:33 GMT
X-Forwarded-For: 18.246.31.226
X-Forwarded-Proto: https
X-Hook-Uuid: 153b4c79-79fa-412c-9380-e2d25ecde689
X-Request-Uuid: 44a14708-0bc3-4ef6-8447-09dfdbe9261d
Payload
{
    "push": {
        "changes": [
            {
                "old": {
                    "name": "main",
                    "target": {
                        "type": "commit",
                        "hash": "f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72",
                        "date": "2022-10-26T09:57:22+00:00",
                        "author": {
                            "type": "author",
                            "raw": "Antoine Lambert <anlambert@softwareheritage.org>",
                            "user": {
                                "display_name": "Antoine Lambert",
                                "links": {
                                    "self": {
                                        "href": "https://api.bitbucket.org/2.0/users/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D"
                                    },
                                    "avatar": {
                                        "href": "https://secure.gravatar.com/avatar/d522e4a403af0605784d0f7936a506a3?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAL-1.png"
                                    },
                                    "html": {
                                        "href": "https://bitbucket.org/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D/"
                                    }
                                },
                                "type": "user",
                                "uuid": "{65f18e6c-2e1c-4a89-99c1-f7bad58d5a39}",
                                "account_id": "5d1483f6f46aa30c271c968e",
                                "nickname": "Antoine Lambert"
                            }
                        },
                        "message": "Initial commit",
                        "summary": {
                            "type": "rendered",
                            "raw": "Initial commit",
                            "markup": "markdown",
                            "html": "<p>Initial commit</p>"
                        },
                        "links": {
                            "self": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                            },
                            "html": {
                                "href": "https://bitbucket.org/anlambert/webhook-test/commits/f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                            }
                        },
                        "parents": [],
                        "rendered": {},
                        "properties": {}
                    },
                    "links": {
                        "self": {
                            "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/refs/branches/main"
                        },
                        "commits": {
                            "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commits/main"
                        },
                        "html": {
                            "href": "https://bitbucket.org/anlambert/webhook-test/branch/main"
                        }
                    },
                    "type": "branch",
                    "merge_strategies": [
                        "merge_commit",
                        "squash",
                        "fast_forward"
                    ],
                    "default_merge_strategy": "merge_commit"
                },
                "new": {
                    "name": "main",
                    "target": {
                        "type": "commit",
                        "hash": "45ed1aeeb43008b0ec5666ef67242f66639920d7",
                        "date": "2022-10-26T10:03:24+00:00",
                        "author": {
                            "type": "author",
                            "raw": "Antoine Lambert <anlambert@softwareheritage.org>",
                            "user": {
                                "display_name": "Antoine Lambert",
                                "links": {
                                    "self": {
                                        "href": "https://api.bitbucket.org/2.0/users/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D"
                                    },
                                    "avatar": {
                                        "href": "https://secure.gravatar.com/avatar/d522e4a403af0605784d0f7936a506a3?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAL-1.png"
                                    },
                                    "html": {
                                        "href": "https://bitbucket.org/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D/"
                                    }
                                },
                                "type": "user",
                                "uuid": "{65f18e6c-2e1c-4a89-99c1-f7bad58d5a39}",
                                "account_id": "5d1483f6f46aa30c271c968e",
                                "nickname": "Antoine Lambert"
                            }
                        },
                        "message": "Add new line in README to trigger webhook when pushing commit\n",
                        "summary": {
                            "type": "rendered",
                            "raw": "Add new line in README to trigger webhook when pushing commit\n",
                            "markup": "markdown",
                            "html": "<p>Add new line in README to trigger webhook when pushing commit</p>"
                        },
                        "links": {
                            "self": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/45ed1aeeb43008b0ec5666ef67242f66639920d7"
                            },
                            "html": {
                                "href": "https://bitbucket.org/anlambert/webhook-test/commits/45ed1aeeb43008b0ec5666ef67242f66639920d7"
                            }
                        },
                        "parents": [
                            {
                                "type": "commit",
                                "hash": "f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72",
                                "links": {
                                    "self": {
                                        "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                                    },
                                    "html": {
                                        "href": "https://bitbucket.org/anlambert/webhook-test/commits/f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                                    }
                                }
                            }
                        ],
                        "rendered": {},
                        "properties": {}
                    },
                    "links": {
                        "self": {
                            "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/refs/branches/main"
                        },
                        "commits": {
                            "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commits/main"
                        },
                        "html": {
                            "href": "https://bitbucket.org/anlambert/webhook-test/branch/main"
                        }
                    },
                    "type": "branch",
                    "merge_strategies": [
                        "merge_commit",
                        "squash",
                        "fast_forward"
                    ],
                    "default_merge_strategy": "merge_commit"
                },
                "truncated": false,
                "created": false,
                "forced": false,
                "closed": false,
                "links": {
                    "commits": {
                        "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commits?include=45ed1aeeb43008b0ec5666ef67242f66639920d7&exclude=f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                    },
                    "diff": {
                        "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/diff/45ed1aeeb43008b0ec5666ef67242f66639920d7..f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                    },
                    "html": {
                        "href": "https://bitbucket.org/anlambert/webhook-test/branches/compare/45ed1aeeb43008b0ec5666ef67242f66639920d7..f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                    }
                },
                "commits": [
                    {
                        "type": "commit",
                        "hash": "45ed1aeeb43008b0ec5666ef67242f66639920d7",
                        "date": "2022-10-26T10:03:24+00:00",
                        "author": {
                            "type": "author",
                            "raw": "Antoine Lambert <anlambert@softwareheritage.org>",
                            "user": {
                                "display_name": "Antoine Lambert",
                                "links": {
                                    "self": {
                                        "href": "https://api.bitbucket.org/2.0/users/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D"
                                    },
                                    "avatar": {
                                        "href": "https://secure.gravatar.com/avatar/d522e4a403af0605784d0f7936a506a3?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAL-1.png"
                                    },
                                    "html": {
                                        "href": "https://bitbucket.org/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D/"
                                    }
                                },
                                "type": "user",
                                "uuid": "{65f18e6c-2e1c-4a89-99c1-f7bad58d5a39}",
                                "account_id": "5d1483f6f46aa30c271c968e",
                                "nickname": "Antoine Lambert"
                            }
                        },
                        "message": "Add new line in README to trigger webhook when pushing commit\n",
                        "summary": {
                            "type": "rendered",
                            "raw": "Add new line in README to trigger webhook when pushing commit\n",
                            "markup": "markdown",
                            "html": "<p>Add new line in README to trigger webhook when pushing commit</p>"
                        },
                        "links": {
                            "self": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/45ed1aeeb43008b0ec5666ef67242f66639920d7"
                            },
                            "html": {
                                "href": "https://bitbucket.org/anlambert/webhook-test/commits/45ed1aeeb43008b0ec5666ef67242f66639920d7"
                            },
                            "diff": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/diff/45ed1aeeb43008b0ec5666ef67242f66639920d7"
                            },
                            "approve": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/45ed1aeeb43008b0ec5666ef67242f66639920d7/approve"
                            },
                            "comments": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/45ed1aeeb43008b0ec5666ef67242f66639920d7/comments"
                            },
                            "statuses": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/45ed1aeeb43008b0ec5666ef67242f66639920d7/statuses"
                            },
                            "patch": {
                                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/patch/45ed1aeeb43008b0ec5666ef67242f66639920d7"
                            }
                        },
                        "parents": [
                            {
                                "type": "commit",
                                "hash": "f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72",
                                "links": {
                                    "self": {
                                        "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test/commit/f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                                    },
                                    "html": {
                                        "href": "https://bitbucket.org/anlambert/webhook-test/commits/f069b8b3a68b043cdab1cabd2b3415eaf1b0ef72"
                                    }
                                }
                            }
                        ],
                        "rendered": {},
                        "properties": {}
                    }
                ]
            }
        ]
    },
    "repository": {
        "type": "repository",
        "full_name": "anlambert/webhook-test",
        "links": {
            "self": {
                "href": "https://api.bitbucket.org/2.0/repositories/anlambert/webhook-test"
            },
            "html": {
                "href": "https://bitbucket.org/anlambert/webhook-test"
            },
            "avatar": {
                "href": "https://bytebucket.org/ravatar/%7Bb5d2882b-effe-4cf8-bb5a-8a757bdbfe28%7D?ts=default"
            }
        },
        "name": "webhook-test",
        "scm": "git",
        "website": null,
        "owner": {
            "display_name": "Antoine Lambert",
            "links": {
                "self": {
                    "href": "https://api.bitbucket.org/2.0/users/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D"
                },
                "avatar": {
                    "href": "https://secure.gravatar.com/avatar/d522e4a403af0605784d0f7936a506a3?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAL-1.png"
                },
                "html": {
                    "href": "https://bitbucket.org/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D/"
                }
            },
            "type": "user",
            "uuid": "{65f18e6c-2e1c-4a89-99c1-f7bad58d5a39}",
            "account_id": "5d1483f6f46aa30c271c968e",
            "nickname": "Antoine Lambert"
        },
        "workspace": {
            "type": "workspace",
            "uuid": "{65f18e6c-2e1c-4a89-99c1-f7bad58d5a39}",
            "name": "Antoine Lambert",
            "slug": "anlambert",
            "links": {
                "avatar": {
                    "href": "https://bitbucket.org/workspaces/anlambert/avatar/?ts=1561625644"
                },
                "html": {
                    "href": "https://bitbucket.org/anlambert/"
                },
                "self": {
                    "href": "https://api.bitbucket.org/2.0/workspaces/anlambert"
                }
            }
        },
        "is_private": false,
        "project": {
            "type": "project",
            "key": "WEB",
            "uuid": "{4c11da16-a545-45c4-84c2-d0ba937a2e23}",
            "name": "webhook-test",
            "links": {
                "self": {
                    "href": "https://api.bitbucket.org/2.0/workspaces/anlambert/projects/WEB"
                },
                "html": {
                    "href": "https://bitbucket.org/anlambert/workspace/projects/WEB"
                },
                "avatar": {
                    "href": "https://bitbucket.org/account/user/anlambert/projects/WEB/avatar/32?ts=1666778241"
                }
            }
        },
        "uuid": "{b5d2882b-effe-4cf8-bb5a-8a757bdbfe28}"
    },
    "actor": {
        "display_name": "Antoine Lambert",
        "links": {
            "self": {
                "href": "https://api.bitbucket.org/2.0/users/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D"
            },
            "avatar": {
                "href": "https://secure.gravatar.com/avatar/d522e4a403af0605784d0f7936a506a3?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FAL-1.png"
            },
            "html": {
                "href": "https://bitbucket.org/%7B65f18e6c-2e1c-4a89-99c1-f7bad58d5a39%7D/"
            }
        },
        "type": "user",
        "uuid": "{65f18e6c-2e1c-4a89-99c1-f7bad58d5a39}",
        "account_id": "5d1483f6f46aa30c271c968e",
        "nickname": "Antoine Lambert"
    }
}
anlambert added a revision: D8798: save_origin_webhooks: Add push webhook receivers for popular forges.Oct 31 2022, 5:14 PM
anlambert added a commit: rDWAPPS93696b9cc7b9: save_origin_webhooks: Add push webhook receivers for popular forges.Nov 3 2022, 10:51 AM
anlambert added a comment.Nov 8 2022, 4:57 PM

Webhooks feature has been deployed and activated on staging, now let's test it before activating it for production too.

anlambert added a comment.Nov 8 2022, 5:06 PM

GitHub case: I added webhook settings for that sample repository of mine https://github.com/anlambert/webhook-test using the following payload URL https://webapp.staging.swh.network/api/1/origin/save/webhook/github/.

After pushing a commit, the payload was successfully received and the repository archived.

anlambert added a comment.Nov 8 2022, 5:25 PM

GitLab case: I added webhook settings for that sample repository of mine https://gitlab.com/anlambert/test using the following payload URL https://webapp.staging.swh.network/api/1/origin/save/webhook/gitlab/.

After pushing a commit, the payload was successfully received and the repository scheduled for archival.

anlambert added a comment.Nov 8 2022, 5:27 PM

Gitea case: I added webhook settings for that sample repository of mine https://try.gitea.io/anlambert/webhook-test.git using the following payload URL https://webapp.staging.swh.network/api/1/origin/save/webhook/gitea/.

After pushing a commit, the payload was successfully received and the repository scheduled for archival.

anlambert added a comment.Nov 8 2022, 5:29 PM

BitBucket case: I added webhook settings for that sample repository of mine https://bitbucket.org/anlambert/webhook-test.git using the following payload URL https://webapp.staging.swh.network/api/1/origin/save/webhook/bitbucket/.

After pushing a commit, the payload was successfully received and the repository scheduled for archival.

anlambert added a comment.Nov 8 2022, 5:41 PM

SourceForge case: I added webhook settings for that sample repositories of mine :

using the following payload URL https://webapp.staging.swh.network/api/1/origin/save/webhook/sourceforge/.

After pushing commits, the payloads were successfully received and the repositories scheduled for archival.

anlambert added a comment.Nov 8 2022, 5:49 PM

All forge webhook receivers are working as expected, time to activate them in production.

https://gitlab.softwareheritage.org/infra/puppet/puppet-swh-site/-/merge_requests/578

anlambert added a comment.Nov 8 2022, 6:03 PM

New forge webhooks now deployed to production, see Request archival section of Web API endpoints.

I am leaving that task opened as that new feature needs to be advertised through a blog post.

anlambert added a revision: D8841: save_code_now: Allow request creation if previous for origin is running.Nov 14 2022, 5:27 PM
anlambert added a commit: rDWAPPSab849109ea1d: save_code_now: Allow request creation if previous for origin is running.Nov 16 2022, 3:29 PM
anlambert added a revision: D8867: save_origin_webhooks: Handle github ping event.Nov 21 2022, 4:46 PM
anlambert added a revision: D8868: save_origin_webhooks: Handle private repositories.Nov 21 2022, 5:40 PM
anlambert added a commit: rDWAPPS84922dfd698b: save_origin_webhooks: Handle github ping event.Nov 22 2022, 10:39 AM
anlambert added a commit: rDWAPPS1b8992d585a0: save_origin_webhooks: Handle private repositories.
anlambert added a revision: D8889: save_code_now: Add from_webhook field to SaveOriginRequest model.Nov 28 2022, 2:09 PM
anlambert added a revision: D8890: metrics: Add gauge counting save origin requests created from webhooks.Nov 28 2022, 3:17 PM
anlambert added a revision: D8902: save_code_now: Indicate if request was created from webhook in Web UI.Nov 29 2022, 5:00 PM
anlambert added a commit: rDWAPPS12a291957fa0: save_code_now: Add webhook info fields to SaveOriginRequest model.Nov 30 2022, 4:22 PM
anlambert added a commit: rDWAPPSb4fc49b3fe92: metrics: Add gauge counting save origin requests created from webhooks.
anlambert added a commit: rDWAPPSce8da30b9842: metrics/tests/test_app: Fix test name.
anlambert added a commit: rDWAPPSf392e87cf449: save_code_now: Indicate if request was created from webhook in Web UI.
moranegg added a subscriber: moranegg.Dec 6 2022, 11:22 PM

This task seems rich with information. Is this adapted for a guide or tutorial?
I'm creating a task for a tutorial about the webhook following Ambassador Pierre Poulain suggestion.

moranegg mentioned this in T4717: Create tutorial for using the webhook for a Save Code Now .Dec 6 2022, 11:24 PM
gitlab-migration closed this task as Migrated.Jan 8 2023, 5:05 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal