Page MenuHomeSoftware Heritage
Differential D8800

save_origin_webhooks, class edition
AbandonedPublicDraft
Actions

Authored by vlorentz on Nov 2 2022, 10:23 AM.

Details

Reviewers
anlambert
Summary

This refactors the webhook API in D8798, using classes instead of simple functions

This removes most of the boilerplate

Diff Detail

Repository
rDWAPPS Web applications
Branch
webhook-classes
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 32661
Build 51167: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 51166: arc lint + arc unit

Unit TestsFailed
View All

TimeTest
20,097 msJenkins > should contain argument with language in url::Tests / Cypress tests / Test Content Language Select When language is switched from dropdown should contain argument with language in url
AssertionError: Timed out retrying after 20000ms: expected '?origin_url=https://github.com/memononen/libtess2&path=premake4.lua' to include 'language=lua' at Context.eval (webpack:///./cypress/e2e/language-select.cy.js:60:9)
531 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.add_forge_now.test_api_views::Tests / Python tests / test_add_forge_request_create_anonymous_user
361 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.add_forge_now.test_api_views::Tests / Python tests / test_add_forge_request_create_duplicate
240 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.add_forge_now.test_api_views::Tests / Python tests / test_add_forge_request_create_empty
363 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.add_forge_now.test_api_views::Tests / Python tests / test_add_forge_request_create_invalid_forge_url
View Full Test Results (1 Failed · 1,066 Passed · 2 Skipped)

Event Timeline

vlorentz updated this revision to Diff 31712.Nov 2 2022, 10:23 AM
vlorentz created this revision.
  • remove unused set_docstring
vlorentz mentioned this in D8798: save_origin_webhooks: Add push webhook receivers for popular forges.Nov 2 2022, 10:27 AM
swh-public-ci added a comment.Nov 2 2022, 10:33 AM

Build is green

Patch application report for D8800 (id=31711)

Rebasing onto 024da72220...

Current branch diff-target is up to date.
Changes applied before test
commit ae5f6a44cb83e93be5deb5767c9702baddde2da4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Nov 2 10:22:24 2022 +0100

    Refactor webhook API using classes instead of simple functions
    
    This removes most of the boilerplate

commit ad5ce708c1d00251bf99f78b5a556b79ae519cc9
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Nov 2 09:44:06 2022 +0100

    save_origin_webhooks: Add push webhook receivers for popular forges
    
    Summary:
    Add new Web API endpoints accepting only POST requests coming from push
    webhooks of the following popular forges or their instances:
    
    - Bitbucket
    - Gitea
    - GitHub
    - GitLab
    - SourceForge
    
    There is one API endpoint per forge type enabling to request or update the
    archival of a repository when new commits are pushed to it through the Save
    Code Now service. Each endpoint simply processes the webhook JSON payload
    sent by a forge in order to extract the repository URL and the visit type
    in order to create a new save request for the repository.
    
    Related to T4548
    
    Reviewers: #reviewers
    
    Maniphest Tasks: T4548
    
    Differential Revision: https://forge.softwareheritage.org/D8798

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/2100/ for more details.

Harbormaster completed remote builds in B32660: Diff 31711.Nov 2 2022, 10:33 AM
swh-public-ci added a comment.Nov 2 2022, 10:34 AM

Build has FAILED

Patch application report for D8800 (id=31712)

Rebasing onto 024da72220...

Current branch diff-target is up to date.
Changes applied before test
commit 933f546249ad3fd4f1d6e624c1d841eba1d3a835
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Nov 2 10:24:31 2022 +0100

    remove unused set_docstring

commit ae5f6a44cb83e93be5deb5767c9702baddde2da4
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Nov 2 10:22:24 2022 +0100

    Refactor webhook API using classes instead of simple functions
    
    This removes most of the boilerplate

commit ad5ce708c1d00251bf99f78b5a556b79ae519cc9
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Wed Nov 2 09:44:06 2022 +0100

    save_origin_webhooks: Add push webhook receivers for popular forges
    
    Summary:
    Add new Web API endpoints accepting only POST requests coming from push
    webhooks of the following popular forges or their instances:
    
    - Bitbucket
    - Gitea
    - GitHub
    - GitLab
    - SourceForge
    
    There is one API endpoint per forge type enabling to request or update the
    archival of a repository when new commits are pushed to it through the Save
    Code Now service. Each endpoint simply processes the webhook JSON payload
    sent by a forge in order to extract the repository URL and the visit type
    in order to create a new save request for the repository.
    
    Related to T4548
    
    Reviewers: #reviewers
    
    Maniphest Tasks: T4548
    
    Differential Revision: https://forge.softwareheritage.org/D8798

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/2101/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/2101/console

Harbormaster returned this revision to the author for changes because remote builds failed.Nov 2 2022, 10:34 AM
Harbormaster failed remote builds in B32661: Diff 31712!
vlorentz abandoned this revision.Nov 3 2022, 9:26 AM

Revision Contents
Changeset List

PathSize
swh/
web/
1 line
save_origin_webhooks/
44 lines
114 lines
28 lines
36 lines
34 lines
61 lines
17 lines
settings/
1 line
tests/
6 lines
17 lines
resources/
http_esnode1.internal.softwareheritage.org/
save_code_now/
data/
http_esnode1.internal.softwareheritage.org/
resources/
http_esnode1.internal.softwareheritage.org/
62 lines
save_origin_webhooks/
data/
333 lines
179 lines
185 lines
62 lines
https_sourceforge.net/
81 lines
80 lines
81 lines
36 lines
36 lines
37 lines
88 lines
87 lines
88 lines
88 lines
100 lines
143 lines

Diff 31712

View Options

swh/web/config.py

Loading...
View Options

swh/web/save_origin_webhooks/__init__.py

Loading...
View Options

swh/web/save_origin_webhooks/bitbucket.py

Loading...
View Options

swh/web/save_origin_webhooks/generic_receiver.py

Loading...
View Options

swh/web/save_origin_webhooks/gitea.py

Loading...
View Options

swh/web/save_origin_webhooks/github.py

Loading...
View Options

swh/web/save_origin_webhooks/gitlab.py

Loading...
View Options

swh/web/save_origin_webhooks/sourceforge.py

Loading...
View Options

swh/web/save_origin_webhooks/urls.py

Loading...
View Options

swh/web/settings/tests.py

Loading...
View Options

swh/web/tests/conftest.py

Loading...
View Options

swh/web/tests/helpers.py

Loading...
View Options

swh/web/tests/resources/http_esnode1.internal.softwareheritage.org/swh_workers-*__search

Loading...
View Options

swh/web/tests/save_code_now/data/http_esnode1.internal.softwareheritage.org/swh_workers-*__search

Loading...
View Options

swh/web/tests/save_origin_webhooks/__init__.py

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/bitbucket_webhook_payload.json

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/gitea_webhook_payload.json

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/github_webhook_payload.json

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/gitlab_webhook_payload.json

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/https_sourceforge.net/rest_p_webhook-test-git

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/https_sourceforge.net/rest_p_webhook-test-hg

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/https_sourceforge.net/rest_p_webhook-test-svn

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/sourceforge_webhook_payload_git.json

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/sourceforge_webhook_payload_hg.json

Loading...
View Options

swh/web/tests/save_origin_webhooks/data/sourceforge_webhook_payload_svn.json

Loading...
View Options

swh/web/tests/save_origin_webhooks/test_bitbucket.py

Loading...
View Options

swh/web/tests/save_origin_webhooks/test_gitea.py

Loading...
View Options

swh/web/tests/save_origin_webhooks/test_github.py

Loading...
View Options

swh/web/tests/save_origin_webhooks/test_gitlab.py

Loading...
View Options

swh/web/tests/save_origin_webhooks/test_sourceforge.py

Loading...
View Options

swh/web/tests/save_origin_webhooks/utils.py

Loading...
About · Developer information · Legal