Page MenuHomeSoftware Heritage

loader: Add origin URL and visit type as sentry tags
ClosedPublic

Authored by anlambert on Sep 8 2022, 5:30 PM.

Details

Summary

It enables to search sentry events related to a particular loaded origin
URL or for a given visit type from the sentry Web UI.

See Searchable Properties and Customize Tags in Sentry documentation.

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

swh/loader/core/tests/conftest.py
1 ↗(On Diff #30391)

Copied and pasted from swh-indexer conftest file. It could be moved to a pytest plugin in swh-core to avoid code duplication.

Build is green

Patch application report for D8430 (id=30391)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit b41a92d72654af6d0c83095c0dbce7895bbca087
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/864/ for more details.

vlorentz added a subscriber: vlorentz.

Hmm, I was about to say:

I think it would make more sense to use breadcrumbs here.

I used tags in swh-indexer because it handles origins and objects in batch, so "current origin/object" keeps changing back and forth.

But actually, breadcrumbs are flooded with request logs, so I guess tags are good to make this visible.

This revision is now accepted and ready to land.Sep 8 2022, 5:43 PM

Hmm, I was about to say:

I think it would make more sense to use breadcrumbs here.

I used tags in swh-indexer because it handles origins and objects in batch, so "current origin/object" keeps changing back and forth.

But actually, breadcrumbs are flooded with request logs, so I guess tags are good to make this visible.

I also think that using tags is the right choice here.

We will be able to quickly find related issues and events associated to an origin URL using the sentry search form as you can currently
do with swh-indexer ones.

I just noticed that base class for package loaders override the load method so I should also set sentry tags here, will update.

Update:

  • Move sentry_sdk.set_tag calls in BaseLoader constructor in order for package loaders to also benefit from them in sentry reports
  • Add test for package loader checking tags are correctly sent to sentry
This revision is now accepted and ready to land.Sep 9 2022, 5:05 PM

Remove package loader from changeset

Build has FAILED

Patch application report for D8430 (id=30422)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 010b59e10b541a1d6fe039c0cfb98da0c8eb3410
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/console

Build has FAILED

Patch application report for D8430 (id=30422)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 010b59e10b541a1d6fe039c0cfb98da0c8eb3410
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/console

Build will fail until D8434 gets landed and a new swh-core tag pushed.

Build has FAILED

Patch application report for D8430 (id=30423)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 4a2fe2f861be3b4e2af2cf22fcc20d105a42fc86
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/866/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/866/console

Build is green

Patch application report for D8430 (id=30423)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 4a2fe2f861be3b4e2af2cf22fcc20d105a42fc86
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/867/ for more details.