Page MenuHomeSoftware Heritage
Feed Advanced Search

Sep 13 2022

anlambert updated the diff for D8420: auth: Improve login management and configuration.

Bump swh-auth requirement

Sep 13 2022, 10:58 AM
anlambert closed D8451: django/views: Rename next_path query parameter to next.
Sep 13 2022, 10:49 AM
anlambert committed rDAUTH2b1b97730382: django/views: Rename next_path query parameter to next (authored by anlambert).
django/views: Rename next_path query parameter to next
Sep 13 2022, 10:48 AM
anlambert closed D8452: docs/api/register-account: Update webapp login URLs.
Sep 13 2022, 10:48 AM
anlambert committed rDDEPd9ab93a57daf: docs/api/register-account: Update webapp login URLs (authored by anlambert).
docs/api/register-account: Update webapp login URLs
Sep 13 2022, 10:48 AM

Sep 12 2022

anlambert requested review of D8453: user/loaders: Add new loaders in development to loaders info page.
Sep 12 2022, 5:22 PM
anlambert added a revision to T3117: Publish status of existing listers and loaders: D8453: user/loaders: Add new loaders in development to loaders info page.
Sep 12 2022, 5:08 PM · Documentation, Roadmap 2022, meta-task, Community Building, Roadmap 2021
anlambert requested review of D8452: docs/api/register-account: Update webapp login URLs.
Sep 12 2022, 4:59 PM
anlambert added a comment to D8420: auth: Improve login management and configuration.

However, that parameter can be omitted in that case so we can safely rename it and I will update that doc.

Sep 12 2022, 4:53 PM
anlambert updated the diff for D8420: auth: Improve login management and configuration.

Use next as redirect field name (default in django.contrib.auth).

Sep 12 2022, 4:52 PM
anlambert requested review of D8451: django/views: Rename next_path query parameter to next.
Sep 12 2022, 4:47 PM
anlambert added a comment to D8420: auth: Improve login management and configuration.
In D8420#219455, @olasd wrote:

Is there anything preventing swh-auth from using next= instead of next_path=?

I do not think we shared such URLs in the wild as they are used internally during login process
so we could rename the query parameter to next in swh-auth yes, that would be simpler for sure.

Sep 12 2022, 4:39 PM
anlambert added a comment to D8420: auth: Improve login management and configuration.
In D8420#219455, @olasd wrote:

Is there anything preventing swh-auth from using next= instead of next_path=?

Sep 12 2022, 4:23 PM
anlambert added a comment to D8420: auth: Improve login management and configuration.

Hmm, that's not a setting, your change is overriding a module-level variable (which may be fine? but may not be? Really depends on import ordering and whether code in settings.py is run before other modules import django.contrib.auth).

Sep 12 2022, 4:03 PM
anlambert added a comment to D8420: auth: Improve login management and configuration.
In D8420#219363, @olasd wrote:

While you're refactoring this, is there a reason for not using the default REDIRECT_FIELD_NAME from django.contrib.auth ("next") for the login/logout views, instead of "next_path"? This would save us from having to override it on every user_passes_test call.

Oh I missed that django setting, let me check.

Sep 12 2022, 3:16 PM
anlambert updated the diff for D8420: auth: Improve login management and configuration.

Prefer to override django.contrib.auth.REDIRECT_FIELD_NAME instead of
using redirect_field_name parameter in auth related decorators.

Sep 12 2022, 3:15 PM
anlambert closed D8444: tests/conftest: Remove sentry fixtures.
Sep 12 2022, 2:54 PM
anlambert committed rDCIDXe25a2f4e4a34: tests/conftest: Remove sentry fixtures (authored by anlambert).
tests/conftest: Remove sentry fixtures
Sep 12 2022, 2:54 PM
anlambert added a comment to D8420: auth: Improve login management and configuration.
In D8420#219363, @olasd wrote:

While you're refactoring this, is there a reason for not using the default REDIRECT_FIELD_NAME from django.contrib.auth ("next") for the login/logout views, instead of "next_path"? This would save us from having to override it on every user_passes_test call.

Sep 12 2022, 2:50 PM
anlambert updated the diff for D8444: tests/conftest: Remove sentry fixtures.

Bump swh.core requirement

Sep 12 2022, 2:42 PM
anlambert accepted D8426: Return none for missing snapshot in a visit status.

Looks good to me.

Sep 12 2022, 2:32 PM
anlambert requested review of D8445: query_language/tokens: Declare new visit types.
Sep 12 2022, 2:03 PM
anlambert closed D8401: user/listers: Add new listers in development to listers info page.
Sep 12 2022, 2:02 PM
anlambert committed rDDOC6edd44eab8ca: user/listers: Add new listers in development to listers info page (authored by anlambert).
user/listers: Add new listers in development to listers info page
Sep 12 2022, 2:02 PM
anlambert added inline comments to D8426: Return none for missing snapshot in a visit status.
Sep 12 2022, 1:35 PM
anlambert closed D8430: loader: Add origin URL and visit type as sentry tags.
Sep 12 2022, 12:31 PM
anlambert committed rDLDBASE4a2fe2f861be: loader: Add origin URL and visit type as sentry tags (authored by anlambert).
loader: Add origin URL and visit type as sentry tags
Sep 12 2022, 12:31 PM
anlambert requested review of D8444: tests/conftest: Remove sentry fixtures.
Sep 12 2022, 12:18 PM
anlambert closed D8434: pytest_plugin: Add sentry_events fixture from swh-indexer.
Sep 12 2022, 11:46 AM
anlambert committed rDCORE5a898b2d5650: pytest_plugin: Add sentry_events fixture from swh-indexer (authored by anlambert).
pytest_plugin: Add sentry_events fixture from swh-indexer
Sep 12 2022, 11:46 AM
anlambert added inline comments to D8434: pytest_plugin: Add sentry_events fixture from swh-indexer.
Sep 12 2022, 11:43 AM
anlambert updated the diff for D8434: pytest_plugin: Add sentry_events fixture from swh-indexer.

Update:

  • rename some fixtures
  • remove unneeded monkeypatch fixture use
Sep 12 2022, 11:41 AM

Sep 9 2022

anlambert added a comment to D8430: loader: Add origin URL and visit type as sentry tags.

Build has FAILED

Patch application report for D8430 (id=30422)

Rebasing onto 6cdf6d3004...

Current branch diff-target is up to date.
Changes applied before test
commit 010b59e10b541a1d6fe039c0cfb98da0c8eb3410
Author: Antoine Lambert <anlambert@softwareheritage.org>
Date:   Thu Sep 8 17:21:01 2022 +0200

    loader: Add origin URL and visit type as sentry tags
    
    It enables to search sentry events related to a particular loaded origin
    URL or for a given visit type from the sentry Web UI.

Link to build: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/
See console output for more information: https://jenkins.softwareheritage.org/job/DLDBASE/job/tests-on-diff/865/console

Sep 9 2022, 5:09 PM
anlambert updated the diff for D8430: loader: Add origin URL and visit type as sentry tags.

Remove package loader from changeset

Sep 9 2022, 5:08 PM
anlambert updated the diff for D8430: loader: Add origin URL and visit type as sentry tags.

Update:

  • Move sentry_sdk.set_tag calls in BaseLoader constructor in order for package loaders to also benefit from them in sentry reports
  • Add test for package loader checking tags are correctly sent to sentry
Sep 9 2022, 5:05 PM
anlambert updated the diff for D8401: user/listers: Add new listers in development to listers info page.

Rebase

Sep 9 2022, 4:51 PM
anlambert added a revision to T3117: Publish status of existing listers and loaders: D8401: user/listers: Add new listers in development to listers info page.
Sep 9 2022, 4:50 PM · Documentation, Roadmap 2022, meta-task, Community Building, Roadmap 2021
anlambert updated the summary of D8401: user/listers: Add new listers in development to listers info page.
Sep 9 2022, 4:50 PM
anlambert updated the diff for D8401: user/listers: Add new listers in development to listers info page.

Add info about other listers planned to be implemented or currently implemented:

Sep 9 2022, 4:48 PM
anlambert renamed T4519: Create a Puppet Forge lister from Create a Puppet Lister to Create a Puppet Forge lister.
Sep 9 2022, 4:21 PM · Puppet lister
anlambert added projects to T1002: ingest Hackage, the Haskell package repository (meta task): Hackage lister, Hackage loader.
Sep 9 2022, 4:00 PM · Hackage loader, Hackage lister, Archive coverage
anlambert added projects to T4494: ingest hackage.haskell.org, the Haskell package repository : Hackage lister, Hackage loader.
Sep 9 2022, 4:00 PM · Hackage loader, Hackage lister, Archive coverage
anlambert created Hackage loader.
Sep 9 2022, 3:59 PM
anlambert created Hackage lister.
Sep 9 2022, 3:58 PM
anlambert added a project to T2833: cpan.loader - archive Perl modules from CPAN: CPAN lister.
Sep 9 2022, 3:52 PM · CPAN lister, Archive coverage
anlambert edited projects for T1777: Rubygems Lister, added: RubyGems lister; removed Lister.
Sep 9 2022, 3:15 PM · RubyGems lister, Archive coverage
anlambert added a project to T4519: Create a Puppet Forge lister: Puppet lister.
Sep 9 2022, 3:15 PM · Puppet lister
anlambert updated subscribers of T4519: Create a Puppet Forge lister.
Sep 9 2022, 3:14 PM · Puppet lister
anlambert added a project to T4520: Create a CPAN lister: CPAN lister.
Sep 9 2022, 3:14 PM · CPAN lister
anlambert created RubyGems lister.
Sep 9 2022, 3:14 PM
anlambert created Puppet lister.
Sep 9 2022, 3:12 PM
anlambert created CPAN lister.
Sep 9 2022, 3:10 PM
anlambert updated the diff for D8420: auth: Improve login management and configuration.

Rebase

Sep 9 2022, 3:08 PM
anlambert closed D8378: Move static assets source code in related django application folders.
Sep 9 2022, 3:07 PM
anlambert committed rDWAPPSc52158a8d7de: Move static assets source code in related django application folders (authored by anlambert).
Move static assets source code in related django application folders
Sep 9 2022, 3:07 PM
anlambert added a comment to T4514: graphql: object_id must be <class 'bytes'> (got None that is a <class 'NoneType'>).

Snapshot is available only in a visit status with 'full' status.

Sep 9 2022, 2:12 PM · GraphQL API
anlambert added a comment to D8426: Return none for missing snapshot in a visit status.
> I do not get the issue here, using the following changes on top of your diff, it works as expected:
>

It will work as expected in the data part. But there will be an error associated, that we think is unnecessary.
With your changes, it will be a "NoneType' object has no attribute 'object_id" instead of an ObjectError

Could you write a snippet query to reproduce the error ? I still do not get what's wrong here.

Sep 9 2022, 2:11 PM
anlambert added a comment to D8426: Return none for missing snapshot in a visit status.
> I do not get the issue here, using the following changes on top of your diff, it works as expected:
>

It will work as expected in the data part. But there will be an error associated, that we think is unnecessary.
With your changes, it will be a "NoneType' object has no attribute 'object_id" instead of an ObjectError

Sep 9 2022, 2:08 PM
anlambert added a comment to D8426: Return none for missing snapshot in a visit status.

Why have two different types, instead of making the snapshot field nullable, like this?

type VisitStatus {
  status: String!
  date: DateTime!
  type: String
  snapshot: Snapshot
}

That is the existing system.
The issue is snapshot has non nullable fields (like SWHID) and GraphQL standard will not allow a
null for an object with non nullable field without an error.

I think it will be a bad idea to remove non-nullable fields from the Sanpshot object.

Sep 9 2022, 1:42 PM
anlambert accepted D8436: pubdev.lister: Decrease verbosity.
Sep 9 2022, 1:32 PM
anlambert accepted D8435: Add revision object as a possible directory entry target.
Sep 9 2022, 11:57 AM
anlambert accepted D8428: npm: Do not generate URIs with spaces in them.
Sep 9 2022, 11:54 AM
anlambert accepted D8396: Add a directoryEntry entrypoint.

Looks good to me.

Sep 9 2022, 11:40 AM
anlambert requested review of D8434: pytest_plugin: Add sentry_events fixture from swh-indexer.
Sep 9 2022, 11:26 AM
anlambert added inline comments to D8396: Add a directoryEntry entrypoint.
Sep 9 2022, 11:22 AM
anlambert planned changes to D8430: loader: Add origin URL and visit type as sentry tags.

Hmm, I was about to say:

I think it would make more sense to use breadcrumbs here.

I used tags in swh-indexer because it handles origins and objects in batch, so "current origin/object" keeps changing back and forth.

But actually, breadcrumbs are flooded with request logs, so I guess tags are good to make this visible.

Sep 9 2022, 10:48 AM
anlambert requested changes to D8396: Add a directoryEntry entrypoint.
Sep 9 2022, 10:37 AM

Sep 8 2022

anlambert requested changes to D8396: Add a directoryEntry entrypoint.
Sep 8 2022, 6:10 PM
anlambert closed D8409: tests: Add some missing unknown object tests.
Sep 8 2022, 5:38 PM
anlambert committed rDGQL62920c3258ab: tests: Add some missing unknown object tests (authored by anlambert).
tests: Add some missing unknown object tests
Sep 8 2022, 5:38 PM
anlambert closed D8408: resolvers: Initialize backends in base classes for node and connection.
Sep 8 2022, 5:38 PM
anlambert committed rDGQL0fc26f83a2a1: resolvers: Initialize backends in base classes for node and connection (authored by anlambert).
resolvers: Initialize backends in base classes for node and connection
Sep 8 2022, 5:38 PM
anlambert closed D8412: webapp: Update configuration for next deployment.
Sep 8 2022, 5:36 PM
anlambert committed rSPSITE9743dd67052b: webapp: Update configuration for next deployment (authored by anlambert).
webapp: Update configuration for next deployment
Sep 8 2022, 5:36 PM
anlambert updated the diff for D8412: webapp: Update configuration for next deployment.

Rebase

Sep 8 2022, 5:36 PM
anlambert requested review of D8430: loader: Add origin URL and visit type as sentry tags.
Sep 8 2022, 5:34 PM
anlambert added inline comments to D8412: webapp: Update configuration for next deployment.
Sep 8 2022, 4:57 PM
anlambert closed D8407: docker/conf: Declare celery tasks for listers/loaders in development.
Sep 8 2022, 2:09 PM
anlambert committed rDENV2aab3ffd23ce: docker/conf: Declare celery tasks for listers/loaders in development (authored by anlambert).
docker/conf: Declare celery tasks for listers/loaders in development
Sep 8 2022, 2:09 PM
anlambert requested review of D8420: auth: Improve login management and configuration.
Sep 8 2022, 2:08 PM
anlambert added a comment to T4512: Strip first level directory when loading tarballs from PyPI?.

But clearly it's not great. I wonder if we could do something about this in swh-web instead

Sep 8 2022, 12:20 PM · PyPI loader
anlambert edited projects for T4511: production: Save Code Now requests for bazaar origins are not scheduled, added: System administration; removed Web app.
Sep 8 2022, 12:00 PM · System administration, Save Code Now
anlambert triaged T4511: production: Save Code Now requests for bazaar origins are not scheduled as Normal priority.
Sep 8 2022, 11:55 AM · System administration, Save Code Now

Sep 7 2022

anlambert added a comment to D8407: docker/conf: Declare celery tasks for listers/loaders in development.

Thanks.

fwiw, i think this should be added with diffs by the contributors of those new listers/loaders.

@bchauvet ^ I expected this to be done already.

@franckbret for your information, i must have forgotten to be explicit about this before so ;)

Sep 7 2022, 7:08 PM
anlambert added a comment to D8338: Hackage: List origins from hackage.haskell.org, The Haskell Package Repository.

@franckbret fyi you have updated the wrong diff (pubdev instead of haskell)

Sep 7 2022, 7:06 PM
anlambert updated the diff for D8407: docker/conf: Declare celery tasks for listers/loaders in development.

Rebase

Sep 7 2022, 7:04 PM
anlambert closed D8413: bin/install: Speedup swh development environment installation.
Sep 7 2022, 7:03 PM
anlambert committed rDENV0e32c5cc1003: bin/install: Speedup swh development environment installation (authored by anlambert).
bin/install: Speedup swh development environment installation
Sep 7 2022, 7:03 PM
anlambert closed D8411: pubdev: Update User-Agent request header value.
Sep 7 2022, 6:59 PM
anlambert committed rDLSc819cc237dca: pubdev: Update User-Agent request header value (authored by anlambert).
pubdev: Update User-Agent request header value
Sep 7 2022, 6:59 PM
anlambert updated the diff for D8378: Move static assets source code in related django application folders.

Rebase

Sep 7 2022, 2:40 PM
anlambert requested review of D8413: bin/install: Speedup swh development environment installation.
Sep 7 2022, 2:30 PM
anlambert committed rDWAPPS33212c44d33f: templates: Move take-new-snapshot.html into save_code_now application (authored by anlambert).
templates: Move take-new-snapshot.html into save_code_now application
Sep 7 2022, 2:17 PM
anlambert requested review of D8412: webapp: Update configuration for next deployment.
Sep 7 2022, 1:52 PM
anlambert added a revision to T4398: Create one django application for each webapp feature: D8412: webapp: Update configuration for next deployment.
Sep 7 2022, 1:52 PM · Web app
anlambert raised the priority of T4239: Unstuck bitbucket incremental lister from Normal to High.

Raising priority as it's been four months since the lister is stuck in production.

Sep 7 2022, 1:30 PM · Bitbucket lister
anlambert requested review of D8411: pubdev: Update User-Agent request header value.
Sep 7 2022, 12:22 PM
anlambert accepted D8410: onboarding/outboarding: mention Keycloak role.
Sep 7 2022, 12:11 PM
anlambert requested review of D8409: tests: Add some missing unknown object tests.
Sep 7 2022, 11:38 AM
anlambert requested review of D8408: resolvers: Initialize backends in base classes for node and connection.
Sep 7 2022, 11:37 AM