Page MenuHomeSoftware Heritage
Differential D8889

save_code_now: Add from_webhook field to SaveOriginRequest model
ClosedPublic
Actions

Authored by anlambert on Nov 28 2022, 2:09 PM.

Details

Reviewers
ardumont
Group Reviewers
Reviewers
Maniphest Tasks
T4548: Add a public API endpoint and documentation to trigger Save Code Now from webhook
Commits
rDWAPPS12a291957fa0: save_code_now: Add webhook info fields to SaveOriginRequest model
Summary

It enables to distinguish which requests were created from a forge
webhook receiver.

Related to T4548

Diff Detail

Repository
rDWAPPS Web applications
Branch
save-code-now-webhook-info
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 33020
Build 51750: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 51749: arc lint + arc unit

Event Timeline

anlambert created this revision.Nov 28 2022, 2:09 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptNov 28 2022, 2:09 PM
swh-public-ci added a comment.Nov 28 2022, 2:26 PM

Build was aborted

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/31/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/31/console

Harbormaster returned this revision to the author for changes because remote builds failed.Nov 28 2022, 2:26 PM
Harbormaster failed remote builds in B32990: Diff 32041!
swh-public-ci added a comment.Nov 28 2022, 2:41 PM

Build was aborted

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/32/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/32/console

swh-public-ci added a comment.Nov 28 2022, 2:52 PM

Build was aborted

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/33/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/33/console

swh-public-ci added a comment.Nov 28 2022, 3:04 PM

Build is green

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/34/ for more details.

swh-public-ci added a comment.Nov 28 2022, 3:15 PM

Build is green

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/35/ for more details.

Harbormaster completed remote builds in B32990: Diff 32041.Nov 28 2022, 3:15 PM
anlambert requested review of this revision.Nov 28 2022, 3:15 PM
anlambert added a child revision: D8890: metrics: Add gauge counting save origin requests created from webhooks.Nov 28 2022, 3:17 PM
vlorentz added a subscriber: vlorentz.Nov 28 2022, 3:42 PM

What is it going to be used for?

And would it make sense to store which webhook was used?

ardumont added a subscriber: ardumont.Nov 29 2022, 12:14 PM

What is it going to be used for?

To add usage stats as a grafana board [1].

[1] D8890

And would it make sense to store which webhook was used?

It'd make sense.

swh/web/save_code_now/migrations/0013_saveoriginrequest_from_webhook.py
19 ↗(On Diff #32041)

Maybe make that an entry which can be used both to determine if a webhook is used and what webhook is actually used?

anlambert added a comment.Nov 29 2022, 12:26 PM

What is it going to be used for?

To compute some stats about usage of that feature.

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

swh/web/save_code_now/migrations/0013_saveoriginrequest_from_webhook.py
19 ↗(On Diff #32041)

No need, we can guess it from the origin URL.

anlambert added a comment.Nov 29 2022, 1:21 PM
In D8889#231299, @anlambert wrote:

What is it going to be used for?

To compute some stats about usage of that feature.

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

Thinking back about it, I can compute the netloc of the origin URL and add it as label in the prometheus metric, will update D8890 then.

vlorentz added a comment.Nov 29 2022, 1:34 PM
In D8889#231299, @anlambert wrote:

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

How? That works for Bitbucket and GitHub, but you can't tell whether "git.example.org" is Gitlab or Gitea

anlambert added a comment.Nov 29 2022, 2:02 PM
In D8889#231323, @vlorentz wrote:
In D8889#231299, @anlambert wrote:

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

How? That works for Bitbucket and GitHub, but you can't tell whether "git.example.org" is Gitlab or Gitea

Indeed and I cannot know the netlocs in advance for the prometheus labels so we really need to add another column in the table with the webhook forge type.

anlambert updated this revision to Diff 32069.Nov 29 2022, 3:03 PM

Update: also add column in save_origin_requests table indicating which forge type sent the webhook

olasd added a subscriber: olasd.Nov 29 2022, 3:11 PM
olasd added inline comments.
swh/web/save_code_now/origin_save.py
430

missing webhook_origin doc

swh-public-ci added a comment.Nov 29 2022, 3:14 PM

Build is green

Patch application report for D8889 (id=32069)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit 08a2f3dc54122a7683d6d35c8c9f5db076f0b121
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/37/ for more details.

Harbormaster completed remote builds in B33016: Diff 32069.Nov 29 2022, 3:15 PM
anlambert updated this revision to Diff 32072.Nov 29 2022, 3:30 PM

Fix docstring

swh-public-ci added a comment.Nov 29 2022, 3:53 PM

Build is green

Patch application report for D8889 (id=32072)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit 4bf0cd9bd08c75c083625e76c503708cb3c759f9
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/39/ for more details.

Harbormaster completed remote builds in B33020: Diff 32072.Nov 29 2022, 3:53 PM
anlambert updated this revision to Diff 32077.Nov 29 2022, 4:31 PM

Update apidoc for /origin/save/ endpoint with new webhook related fields

swh-public-ci added a comment.Nov 29 2022, 4:46 PM

Build is green

Patch application report for D8889 (id=32077)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit fb878e1a2cb5c041b1d3250701c273915fa3a2b6
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/41/ for more details.

Harbormaster completed remote builds in B33027: Diff 32077.Nov 29 2022, 4:46 PM
ardumont accepted this revision.Nov 30 2022, 9:41 AM
This revision is now accepted and ready to land.Nov 30 2022, 9:41 AM
anlambert updated this revision to Diff 32101.Nov 30 2022, 3:39 PM

Fix docstring and doc build.

swh-public-ci added a comment.Nov 30 2022, 3:55 PM

Build is green

Patch application report for D8889 (id=32101)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit 12a291957fa06e3b6fde1ebd3a65cad8e3b03016
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/47/ for more details.

Harbormaster completed remote builds in B33053: Diff 32101.Nov 30 2022, 3:55 PM
Closed by commit rDWAPPS12a291957fa0: save_code_now: Add webhook info fields to SaveOriginRequest model (authored by anlambert). · Explain WhyNov 30 2022, 4:22 PM
This revision was automatically updated to reflect the committed changes.
anlambert added a commit: rDWAPPS12a291957fa0: save_code_now: Add webhook info fields to SaveOriginRequest model.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
swh/
web/
save_code_now/
migrations/
26 lines100%
Loading...
4 lines95%
Loading...
10 lines84%
Loading...
tests/
21 lines100%
Loading...
2 lines95%
Loading...
save_origin_webhooks/
5 lines91%
Loading...
tests/
6 lines100%
Loading...
utils/
4 lines100%
Loading...

Diff 32072

View Options

swh/web/save_code_now/migrations/0013_saveoriginrequest_webhook_info.py

Loading...
View Options

swh/web/save_code_now/models.py

Loading...
View Options

swh/web/save_code_now/origin_save.py

Loading...
View Options

swh/web/save_code_now/tests/test_migrations.py

Loading...
View Options

swh/web/save_code_now/tests/test_origin_save.py

Loading...
View Options

swh/web/save_origin_webhooks/generic_receiver.py

Loading...
View Options

swh/web/save_origin_webhooks/tests/utils.py

Loading...
View Options

swh/web/utils/typing.py

Loading...
About · Developer information · Legal