Page MenuHomeSoftware Heritage
Differential D8430

loader: Add origin URL and visit type as sentry tags
ClosedPublic
Actions

Authored by anlambert on Sep 8 2022, 5:30 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDLDBASE4a2fe2f861be: loader: Add origin URL and visit type as sentry tags
Summary

It enables to search sentry events related to a particular loaded origin
URL or for a given visit type from the sentry Web UI.

See Searchable Properties and Customize Tags in Sentry documentation.

Diff Detail

Repository
rDLDBASE Generic VCS/Package Loader
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

anlambert created this revision.Sep 8 2022, 5:30 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptSep 8 2022, 5:30 PM
anlambert added inline comments.Sep 8 2022, 5:32 PM
swh/loader/core/tests/conftest.py
1 ↗(On Diff #30391)

Copied and pasted from swh-indexer conftest file. It could be moved to a pytest plugin in swh-core to avoid code duplication.

swh-public-ci added a comment.Sep 8 2022, 5:34 PM

Build is green

Patch application report for D8430 (id=30391)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit b41a92d72654af6d0c83095c0dbce7895bbca087
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/864/ for more details.

Harbormaster completed remote builds in B31412: Diff 30391.Sep 8 2022, 5:34 PM
anlambert requested review of this revision.Sep 8 2022, 5:34 PM
vlorentz accepted this revision.Sep 8 2022, 5:43 PM
vlorentz added a subscriber: vlorentz.

Hmm, I was about to say:

I think it would make more sense to use breadcrumbs here.

I used tags in swh-indexer because it handles origins and objects in batch, so "current origin/object" keeps changing back and forth.

But actually, breadcrumbs are flooded with request logs, so I guess tags are good to make this visible.

This revision is now accepted and ready to land.Sep 8 2022, 5:43 PM
anlambert planned changes to this revision.Sep 9 2022, 10:48 AM
In D8430#218927, @vlorentz wrote:

Hmm, I was about to say:

I think it would make more sense to use breadcrumbs here.

I used tags in swh-indexer because it handles origins and objects in batch, so "current origin/object" keeps changing back and forth.

But actually, breadcrumbs are flooded with request logs, so I guess tags are good to make this visible.

I also think that using tags is the right choice here.

We will be able to quickly find related issues and events associated to an origin URL using the sentry search form as you can currently
do with swh-indexer ones.

I just noticed that base class for package loaders override the load method so I should also set sentry tags here, will update.

anlambert mentioned this in D8434: pytest_plugin: Add sentry_events fixture from swh-indexer.Sep 9 2022, 11:24 AM
anlambert updated this revision to Diff 30422.Sep 9 2022, 5:05 PM

Update:

  • Move sentry_sdk.set_tag calls in BaseLoader constructor in order for package loaders to also benefit from them in sentry reports
  • Add test for package loader checking tags are correctly sent to sentry
This revision is now accepted and ready to land.Sep 9 2022, 5:05 PM
anlambert updated this revision to Diff 30423.Sep 9 2022, 5:08 PM

Remove package loader from changeset

swh-public-ci added a comment.Sep 9 2022, 5:09 PM

Build has FAILED

Patch application report for D8430 (id=30422)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 010b59e10b541a1d6fe039c0cfb98da0c8eb3410
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/console

Harbormaster failed remote builds in B31439: Diff 30422!Sep 9 2022, 5:09 PM
anlambert added a comment.Sep 9 2022, 5:09 PM
In D8430#219136, @swh-public-ci wrote:

Build has FAILED

Patch application report for D8430 (id=30422)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 010b59e10b541a1d6fe039c0cfb98da0c8eb3410
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/console

Build will fail until D8434 gets landed and a new swh-core tag pushed.

swh-public-ci added a comment.Sep 9 2022, 5:11 PM

Build has FAILED

Patch application report for D8430 (id=30423)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 4a2fe2f861be3b4e2af2cf22fcc20d105a42fc86
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/866/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/866/console

Harbormaster failed remote builds in B31440: Diff 30423!Sep 9 2022, 5:11 PM
swh-public-ci added a comment.Sep 12 2022, 11:55 AM

Build is green

Patch application report for D8430 (id=30423)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 4a2fe2f861be3b4e2af2cf22fcc20d105a42fc86
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

See https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/867/ for more details.

Harbormaster completed remote builds in B31440: Diff 30423.Sep 12 2022, 11:55 AM
Closed by commit rDLDBASE4a2fe2f861be: loader: Add origin URL and visit type as sentry tags (authored by anlambert). · Explain WhySep 12 2022, 12:31 PM
This revision was automatically updated to reflect the committed changes.
anlambert added a commit: rDLDBASE4a2fe2f861be: loader: Add origin URL and visit type as sentry tags.

Revision Contents
Changeset List

PathSize
swh/
loader/
core/
6 lines
tests/
26 lines
package/
tests/
21 lines

Diff 30442

View Options

swh/loader/core/loader.py

Loading...
View Options

swh/loader/core/tests/test_loader.py

Loading...
View Options

swh/loader/package/tests/test_loader.py

Loading...
About · Developer information · Legal