Page MenuHomeSoftware Heritage
Feed Advanced Search

Sep 19 2022

anlambert requested review of D8504: package/loader: Implement load_status and visit_status.
Sep 19 2022, 4:47 PM
anlambert added a revision to T4124: Golang support: D8505: package/loader: Handle errors when retrieving package version info.
Sep 19 2022, 4:45 PM · Golang loader, Golang lister, Archive coverage
anlambert requested review of D8503: package/tests: Add a visit success test for StubPackageLoader.
Sep 19 2022, 4:44 PM
anlambert closed D8488: jobs/swh-environment: Bump timeout from 90 to 120 minutes.
Sep 19 2022, 1:58 PM
anlambert committed rCJSWH3befb95cf690: jobs/swh-environment: Bump timeout from 90 to 120 minutes (authored by anlambert).
jobs/swh-environment: Bump timeout from 90 to 120 minutes
Sep 19 2022, 1:58 PM
anlambert closed D8498: golang: Update lister name.
Sep 19 2022, 1:58 PM
anlambert committed rDLSfa65f270ed1e: golang: Update lister name (authored by anlambert).
golang: Update lister name
Sep 19 2022, 1:58 PM
anlambert requested review of D8498: golang: Update lister name.
Sep 19 2022, 1:23 PM
anlambert updated the diff for D8495: package/utils: Add debug log and throttling retry to api_info.

Rebase and rename api_info to get_url_body

Sep 19 2022, 11:38 AM
anlambert closed D8496: golang: Ensure to case-encode URLs for retrieving package info.
Sep 19 2022, 10:57 AM
anlambert committed rDLDBASE0c599bdc81e3: golang: Ensure to case-encode URLs for retrieving package info (authored by anlambert).
golang: Ensure to case-encode URLs for retrieving package info
Sep 19 2022, 10:57 AM
anlambert closed D8491: setup.py: Fix debian unstable package build.
Sep 19 2022, 10:54 AM
anlambert committed rDLDCVS83fae9c7b386: setup.py: Fix debian unstable package build (authored by anlambert).
setup.py: Fix debian unstable package build
Sep 19 2022, 10:54 AM
anlambert closed D8490: setup.py: Fix debian unstable package build.
Sep 19 2022, 10:53 AM
anlambert committed rDSEA070f015f4b44: setup.py: Fix debian unstable package build (authored by anlambert).
setup.py: Fix debian unstable package build
Sep 19 2022, 10:53 AM
anlambert updated the diff for D8496: golang: Ensure to case-encode URLs for retrieving package info.

Rebase and update commit message

Sep 19 2022, 10:52 AM
anlambert added a comment to D8495: package/utils: Add debug log and throttling retry to api_info.

api_info sounds like it returns information about an API. Could you rename it to get_url_body?

Sep 19 2022, 10:32 AM

Sep 16 2022

anlambert updated the task description for T4537: Rewrite webapp forms in pure django.
Sep 16 2022, 2:21 PM · Web app
anlambert closed T4398: Create one django application for each webapp feature, a subtask of T3949: Refactor swh.web, as Resolved.
Sep 16 2022, 2:12 PM · meta-task, Roadmap 2022, Web app
anlambert closed T4398: Create one django application for each webapp feature as Resolved.

All swh-web features have been isolated in standalone django applications. This is now deployed so closing this.

Sep 16 2022, 2:12 PM · Web app
anlambert triaged T4537: Rewrite webapp forms in pure django as Normal priority.
Sep 16 2022, 2:10 PM · Web app
anlambert added a subtask for T3949: Refactor swh.web: T4054: Better support for browsers with JS disabled.
Sep 16 2022, 2:03 PM · meta-task, Roadmap 2022, Web app
anlambert added a parent task for T4054: Better support for browsers with JS disabled: T3949: Refactor swh.web.
Sep 16 2022, 2:03 PM · Web app
anlambert requested review of D8496: golang: Ensure to case-encode URLs for retrieving package info.
Sep 16 2022, 1:52 PM
anlambert requested review of D8495: package/utils: Add debug log and throttling retry to api_info.
Sep 16 2022, 1:22 PM

Sep 15 2022

anlambert requested review of D8491: setup.py: Fix debian unstable package build.
Sep 15 2022, 5:40 PM
anlambert updated the diff for D8490: setup.py: Fix debian unstable package build.

Update years in license header

Sep 15 2022, 5:30 PM
anlambert closed D8489: docker: Use swh-search with memory backend as default search engine.
Sep 15 2022, 5:02 PM
anlambert committed rDENV307250792e60: docker: Use swh-search with memory backend as default search engine (authored by anlambert).
docker: Use swh-search with memory backend as default search engine
Sep 15 2022, 5:02 PM
anlambert requested review of D8490: setup.py: Fix debian unstable package build.
Sep 15 2022, 4:57 PM
anlambert added a comment to D8489: docker: Use swh-search with memory backend as default search engine.

I hope the memory usage won't be an issue :/

Sep 15 2022, 4:25 PM
anlambert closed D8487: utils: Cache origin visit types in production webapp only.
Sep 15 2022, 3:56 PM
anlambert committed rDWAPPSd6031c131b73: utils: Cache origin visit types in production webapp only (authored by anlambert).
utils: Cache origin visit types in production webapp only
Sep 15 2022, 3:56 PM
anlambert added a comment to D8489: docker: Use swh-search with memory backend as default search engine.

with a more costly

i think you meant *without* ;)

Nope, when using the docker-compose.search.yml, an elasticsearch service is created, which is clearly more costly than a memory backend.

ah yeah, ok got it.

By default, we are not using that file, hence why you integrated that search rpc service part inside the main docker-compose.
With this ^, We are using the default memory backend.

But if we want to use a real elasticsearch backend, we still can by using -f docker-compose.search.yml which will use a real es container (and backend).

Sep 15 2022, 3:51 PM
anlambert updated subscribers of D8489: docker: Use swh-search with memory backend as default search engine.

ok

I guess the memory implementation is enough to reproduce the same behavior as the actual search service.

Sep 15 2022, 3:50 PM
anlambert added a comment to D8489: docker: Use swh-search with memory backend as default search engine.

with a more costly

i think you meant *without* ;)

Sep 15 2022, 3:46 PM
anlambert closed D8485: api/server: Fix remote search server when using memory backend.
Sep 15 2022, 3:43 PM
anlambert committed rDSEA977938aa114e: api/server: Fix remote search server when using memory backend (authored by anlambert).
api/server: Fix remote search server when using memory backend
Sep 15 2022, 3:43 PM
anlambert requested review of D8489: docker: Use swh-search with memory backend as default search engine.
Sep 15 2022, 3:41 PM
anlambert closed D8486: docker/conf/scheduler: Set no_origins_scheduled_backoff to 1 minute.
Sep 15 2022, 3:39 PM
anlambert committed rDENV68233f9afdb8: docker/conf/scheduler: Set no_origins_scheduled_backoff to 1 minute (authored by anlambert).
docker/conf/scheduler: Set no_origins_scheduled_backoff to 1 minute
Sep 15 2022, 3:39 PM
anlambert retitled D8488: jobs/swh-environment: Bump timeout from 90 to 120 minutes from jobs/swh-environment: Bump timeout from 90 to 120 to jobs/swh-environment: Bump timeout from 90 to 120 minutes.
Sep 15 2022, 3:28 PM
anlambert updated the diff for D8488: jobs/swh-environment: Bump timeout from 90 to 120 minutes.

Update commit message

Sep 15 2022, 3:28 PM
anlambert requested review of D8488: jobs/swh-environment: Bump timeout from 90 to 120 minutes.
Sep 15 2022, 3:27 PM
anlambert requested review of D8487: utils: Cache origin visit types in production webapp only.
Sep 15 2022, 2:57 PM
anlambert requested review of D8486: docker/conf/scheduler: Set no_origins_scheduled_backoff to 1 minute.
Sep 15 2022, 2:38 PM
anlambert requested review of D8485: api/server: Fix remote search server when using memory backend.
Sep 15 2022, 2:36 PM
anlambert closed D8483: pre-commit: Exclude generated files in grpc folder from checks.
Sep 15 2022, 2:22 PM
anlambert committed rDGRPHa426c1d038c0: pre-commit: Exclude generated files in grpc folder from checks (authored by anlambert).
pre-commit: Exclude generated files in grpc folder from checks
Sep 15 2022, 2:22 PM
anlambert closed D8475: recurrent_visits: Allow to set no origins scheduled backoff in config.
Sep 15 2022, 1:55 PM
anlambert committed rDSCHb1afdab92003: recurrent_visits: Allow to set no origins scheduled backoff in config (authored by anlambert).
recurrent_visits: Allow to set no origins scheduled backoff in config
Sep 15 2022, 1:55 PM
anlambert committed rDWAPPS41324ff3b41e: package.json: Upgrade dependencies (authored by anlambert).
package.json: Upgrade dependencies
Sep 15 2022, 11:33 AM
anlambert closed D8463: cypress: Try to fix some flaky tests when executed in parallel.
Sep 15 2022, 11:33 AM
anlambert committed rDWAPPS755755df44b2: cypress: Fix some flaky tests when executed in parallel (authored by anlambert).
cypress: Fix some flaky tests when executed in parallel
Sep 15 2022, 11:33 AM
anlambert requested review of D8483: pre-commit: Exclude generated files in grpc folder from checks.
Sep 15 2022, 11:08 AM
anlambert closed D8476: docker: Add swh-scheduler-update-metrics service.
Sep 15 2022, 11:06 AM
anlambert committed rDENVffd82bc7e0fb: docker: Add swh-scheduler-update-metrics service (authored by anlambert).
docker: Add swh-scheduler-update-metrics service
Sep 15 2022, 11:06 AM
anlambert updated the diff for D8463: cypress: Try to fix some flaky tests when executed in parallel.

Rebase and fix typo in commit message

Sep 15 2022, 11:02 AM
anlambert committed rDWAPPS58fae8e0ea08: webapp/urls: Restore exception handling when fetching history counters (authored by anlambert).
webapp/urls: Restore exception handling when fetching history counters
Sep 15 2022, 11:01 AM
anlambert closed D8474: Fix swh-web 0.1.x execution in docker environment.
Sep 15 2022, 11:01 AM
anlambert committed rDWAPPS13e95bc86cc0: settings/production: Remove login/logout URL settings (authored by anlambert).
settings/production: Remove login/logout URL settings
Sep 15 2022, 11:01 AM
anlambert closed D8453: user/loaders: Add new loaders in development to loaders info page.
Sep 15 2022, 10:51 AM
anlambert committed rDDOC5cef6a88d8e7: user/loaders: Add new loaders in development to loaders info page (authored by anlambert).
user/loaders: Add new loaders in development to loaders info page
Sep 15 2022, 10:51 AM
anlambert committed rDLSf1a1b30fd1fc: arch: Set log level to debug for URL requests (authored by anlambert).
arch: Set log level to debug for URL requests
Sep 15 2022, 10:49 AM
anlambert closed D8458: arch: Use tempfile module to create temporary directory.
Sep 15 2022, 10:49 AM
anlambert committed rDLSa55f171ed5dd: arch: Use tempfile module to create temporary directory (authored by anlambert).
arch: Use tempfile module to create temporary directory
Sep 15 2022, 10:49 AM
anlambert updated the diff for D8474: Fix swh-web 0.1.x execution in docker environment.

Add logger.exception call

Sep 15 2022, 10:48 AM
anlambert updated the summary of D8474: Fix swh-web 0.1.x execution in docker environment.
Sep 15 2022, 10:42 AM
anlambert updated the diff for D8475: recurrent_visits: Allow to set no origins scheduled backoff in config.

Rename global variable

Sep 15 2022, 10:41 AM

Sep 14 2022

anlambert accepted D8477: Bump swh dependencies.
Sep 14 2022, 5:11 PM
anlambert requested review of D8476: docker: Add swh-scheduler-update-metrics service.
Sep 14 2022, 5:03 PM
anlambert requested review of D8475: recurrent_visits: Allow to set no origins scheduled backoff in config.
Sep 14 2022, 4:32 PM
anlambert triaged T4535: "swh scheduler task add" command should print an error message in case of unknown task as Normal priority.
Sep 14 2022, 2:58 PM · Easy hack, Scheduling utilities
anlambert accepted D8473: Add missing parent object types to resolvers.
Sep 14 2022, 2:37 PM
anlambert requested review of D8474: Fix swh-web 0.1.x execution in docker environment.
Sep 14 2022, 2:14 PM
anlambert updated the diff for D8463: cypress: Try to fix some flaky tests when executed in parallel.

Rebase and update commit message

Sep 14 2022, 2:10 PM
anlambert added a comment to D8458: arch: Use tempfile module to create temporary directory.

Don't we actually want to keep it across runs? It's pretty big

Sep 14 2022, 11:31 AM

Sep 13 2022

anlambert accepted D8455: Pubdev: Ensure we correctly parse and sort release versions.

Looks good to me, thanks !

Sep 13 2022, 5:13 PM
anlambert added a comment to D8454: Crates.io: Add last_update for each version of a crate.

Also I think it can be doable to totally remove the GIT part of the lister. The csv files have everything we need. For the incremental part a metadata.json file at the root of the archive a date and a commit hash that represents the date of the db dump.
In incremental case the lister can compare that date to the last update of the crate.

Sep 13 2022, 5:04 PM
anlambert closed D8467: tests/urls: Import test views only when running cypress tests.
Sep 13 2022, 4:37 PM
anlambert committed rDWAPPS5b0ab27a8d62: tests/urls: Import test views only when running cypress tests (authored by anlambert).
tests/urls: Import test views only when running cypress tests
Sep 13 2022, 4:37 PM
anlambert updated the diff for D8463: cypress: Try to fix some flaky tests when executed in parallel.

It seems the issue comes from the use of some before hooks,
turning them to beforeEach ones seems to fix the issue.

Sep 13 2022, 4:35 PM
anlambert requested review of D8467: tests/urls: Import test views only when running cypress tests.
Sep 13 2022, 4:34 PM
anlambert created P1452 (An Untitled Masterwork).
Sep 13 2022, 4:05 PM
anlambert accepted D8462: Add type hints to resolver modules.

Looks good to me, just one missing return type to add.

Sep 13 2022, 3:49 PM
anlambert updated the diff for D8453: user/loaders: Add new loaders in development to loaders info page.
  • Add missing references to Hashbang in arch and aur loader grant
  • Remove duplicated link for crates loader source code
Sep 13 2022, 3:21 PM
anlambert closed D8461: archive_coverage: Add new listed origin types.
Sep 13 2022, 3:09 PM
anlambert committed rDWAPPSa0b384e56f25: archive_coverage: Add new listed origin types (authored by anlambert).
archive_coverage: Add new listed origin types
Sep 13 2022, 3:09 PM
anlambert requested review of D8461: archive_coverage: Add new listed origin types.
Sep 13 2022, 2:51 PM
anlambert requested review of D8463: cypress: Try to fix some flaky tests when executed in parallel.
Sep 13 2022, 2:51 PM
anlambert accepted D8459: GitHub: use correct JSON-LD types for URLs and dates.
Sep 13 2022, 1:41 PM
anlambert closed D8420: auth: Improve login management and configuration.
Sep 13 2022, 1:23 PM
anlambert committed rDWAPPS104f3b79bf3a: auth: Improve login management and configuration (authored by anlambert).
auth: Improve login management and configuration
Sep 13 2022, 1:23 PM
anlambert updated the diff for D8420: auth: Improve login management and configuration.

Rebase

Sep 13 2022, 12:39 PM
anlambert requested review of D8458: arch: Use tempfile module to create temporary directory.
Sep 13 2022, 12:14 PM
anlambert accepted D8457: Add type hints to archive and search modules.

Looks good to me.

Sep 13 2022, 11:14 AM
anlambert closed D8445: query_language/tokens: Declare new visit types.
Sep 13 2022, 11:12 AM
anlambert committed rDSEA445f62d0eeda: query_language/tokens: Declare new visit types (authored by anlambert).
query_language/tokens: Declare new visit types
Sep 13 2022, 11:12 AM
anlambert requested changes to D8455: Pubdev: Ensure we correctly parse and sort release versions.

Thanks, before I can accept this, one inline comment to handle and the packaging dependency needs to be added in the requirements.txt file.

Sep 13 2022, 11:08 AM
anlambert accepted D8456: Ignore empty lines from swh-graph instead of crashing.
Sep 13 2022, 10:59 AM