Page MenuHomeSoftware Heritage

save_code_now: Add from_webhook field to SaveOriginRequest model
ClosedPublic

Authored by anlambert on Nov 28 2022, 2:09 PM.

Details

Summary

It enables to distinguish which requests were created from a forge
webhook receiver.

Related to T4548

Diff Detail

Repository
rDWAPPS Web applications
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Build was aborted

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/31/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/31/console

Harbormaster returned this revision to the author for changes because remote builds failed.Nov 28 2022, 2:26 PM
Harbormaster failed remote builds in B32990: Diff 32041!

Build was aborted

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/32/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/32/console

Build was aborted

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/33/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/33/console

Build is green

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/34/ for more details.

Build is green

Patch application report for D8889 (id=32041)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit ec3dc8b253eddb0cb44a5825d13244f147d1659d
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add from_webhook field to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/35/ for more details.

What is it going to be used for?

And would it make sense to store which webhook was used?

What is it going to be used for?

To add usage stats as a grafana board [1].

[1] D8890

And would it make sense to store which webhook was used?

It'd make sense.

swh/web/save_code_now/migrations/0013_saveoriginrequest_from_webhook.py
19 โ†—(On Diff #32041)

Maybe make that an entry which can be used both to determine if a webhook is used and what webhook is actually used?

What is it going to be used for?

To compute some stats about usage of that feature.

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

swh/web/save_code_now/migrations/0013_saveoriginrequest_from_webhook.py
19 โ†—(On Diff #32041)

No need, we can guess it from the origin URL.

What is it going to be used for?

To compute some stats about usage of that feature.

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

Thinking back about it, I can compute the netloc of the origin URL and add it as label in the prometheus metric, will update D8890 then.

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

How? That works for Bitbucket and GitHub, but you can't tell whether "git.example.org" is Gitlab or Gitea

And would it make sense to store which webhook was used?

It can be easily guessed from the origin URL so I do not think we should add that info.

How? That works for Bitbucket and GitHub, but you can't tell whether "git.example.org" is Gitlab or Gitea

Indeed and I cannot know the netlocs in advance for the prometheus labels so we really need to add another column in the table with the webhook forge type.

Update: also add column in save_origin_requests table indicating which forge type sent the webhook

olasd added inline comments.
swh/web/save_code_now/origin_save.py
430

missing webhook_origin doc

Build is green

Patch application report for D8889 (id=32069)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit 08a2f3dc54122a7683d6d35c8c9f5db076f0b121
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/37/ for more details.

Build is green

Patch application report for D8889 (id=32072)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit 4bf0cd9bd08c75c083625e76c503708cb3c759f9
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/39/ for more details.

Update apidoc for /origin/save/ endpoint with new webhook related fields

Build is green

Patch application report for D8889 (id=32077)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit fb878e1a2cb5c041b1d3250701c273915fa3a2b6
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/41/ for more details.

This revision is now accepted and ready to land.Nov 30 2022, 9:41 AM

Fix docstring and doc build.

Build is green

Patch application report for D8889 (id=32101)

Rebasing onto d9ffdd1759...

Current branch diff-target is up to date.
Changes applied before test
commit 12a291957fa06e3b6fde1ebd3a65cad8e3b03016
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Mon Nov 28 14:07:22 2022 +0100

    save_code_now: Add webhook info fields to SaveOriginRequest model
    
    It enables to distinguish which requests were created from a forge
    webhook receiver and which forge type sent the webhook.
    
    Related to T4548

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/47/ for more details.