Page MenuHomeSoftware Heritage
Feed Advanced Search

Sep 28 2022

anlambert added inline comments to D8560: add_forge_now: Add useful links in requests tables.
Sep 28 2022, 1:54 PM
anlambert requested review of D8560: add_forge_now: Add useful links in requests tables.
Sep 28 2022, 12:06 PM
anlambert added a revision to T4552: Add link to search for origins loaded from an add forge now request: D8560: add_forge_now: Add useful links in requests tables.
Sep 28 2022, 11:57 AM · Web app, Add Forge Now

Sep 27 2022

anlambert added inline comments to D8171: crates: Loader implements incremental mode.
Sep 27 2022, 7:06 PM
anlambert added inline comments to D8454: Crates.io: Add last_update for each version of a crate.
Sep 27 2022, 7:02 PM
anlambert abandoned D8554: arch/aur: Align loader parameters with lister output.

Abandoning this as we must use that format for artifacts metadata apparently.

Sep 27 2022, 6:12 PM
anlambert abandoned D8553: arch/aur: Use dicts instead of lists as extra loader arguments.

Although, as using extra_loader_arguments as an escape hatch to pass lists of artifacts is starting to spread across lister/loader pairs (GNU, Maven, Arch, AUR, CRAN, Crates, and maybe OPAM), I would be fine with defining a new way to pass these objects (as well as arch_metadata/aur_metadata/crates_metadata?), but it should be done for all these loaders and using the same format.

(This will also make it easier to change the storage of these from large JSON objects to rows in the scheduler DB in the future, if their size becomes an issue for postgresql or celery; but not now)

Sep 27 2022, 6:10 PM
anlambert added a comment to D8553: arch/aur: Use dicts instead of lists as extra loader arguments.

My point is that we should not create dozens of undocumented ad-hoc formats in the scheduler database just to remove two lines of code from each loader

Sep 27 2022, 6:03 PM
anlambert added a comment to D8553: arch/aur: Use dicts instead of lists as extra loader arguments.

It avoids the proliferation of ad-hoc formats

Sep 27 2022, 5:51 PM
anlambert added a comment to D8553: arch/aur: Use dicts instead of lists as extra loader arguments.

And so ? This makes the loader simpler and has no effect on what we archive, so better using dicts here.

Sep 27 2022, 5:50 PM
anlambert added a comment to D8553: arch/aur: Use dicts instead of lists as extra loader arguments.
Sep 27 2022, 5:43 PM
anlambert updated the diff for D8535: docker/conf: Declare task queues for new listers and loaders.

Add cpan lister queue

Sep 27 2022, 5:26 PM
anlambert requested review of D8553: arch/aur: Use dicts instead of lists as extra loader arguments.
Sep 27 2022, 5:21 PM
anlambert requested review of D8554: arch/aur: Align loader parameters with lister output.
Sep 27 2022, 5:20 PM
anlambert added a revision to T4233: Ingest Arch Linux: D8554: arch/aur: Align loader parameters with lister output.
Sep 27 2022, 5:16 PM · Arch loader, Arch Lister, Archive coverage
anlambert added a revision to T4466: Ingest Aur repository (Arch User Repository): D8554: arch/aur: Align loader parameters with lister output.
Sep 27 2022, 5:16 PM · AUR loader, AUR lister, Archive coverage
anlambert added a revision to T4466: Ingest Aur repository (Arch User Repository): D8553: arch/aur: Use dicts instead of lists as extra loader arguments.
Sep 27 2022, 5:16 PM · AUR loader, AUR lister, Archive coverage
anlambert added a revision to T4233: Ingest Arch Linux: D8553: arch/aur: Use dicts instead of lists as extra loader arguments.
Sep 27 2022, 5:16 PM · Arch loader, Arch Lister, Archive coverage
anlambert accepted D8528: Nuget: Lister for NuGet the package manager for .NET.
Sep 27 2022, 4:22 PM
anlambert closed D8550: developer-setup: Update packages list and repo URL.
Sep 27 2022, 3:15 PM
anlambert closed D8548: developer-setup: Add instructions to install latest yarn version.
Sep 27 2022, 3:15 PM
anlambert committed rDDOC2824e64cb192: developer-setup: Use https protocol for apt.postgresql.org (authored by anlambert).
developer-setup: Use https protocol for apt.postgresql.org
Sep 27 2022, 3:15 PM
anlambert committed rDDOC4aaa6749a898: developer-setup: Add instructions to install latest yarn version (authored by anlambert).
developer-setup: Add instructions to install latest yarn version
Sep 27 2022, 3:15 PM
anlambert committed rDDOCff12b8c8e9df: developer-setup: Remove postgresql-autodoc from debian packages list (authored by anlambert).
developer-setup: Remove postgresql-autodoc from debian packages list
Sep 27 2022, 3:15 PM
anlambert added a comment to D8542: Cpan: List Perl module origins from cpan.org.

@franckbret , have you considered exploiting the https://fastapi.metacpan.org/v1/release/_search endpoint of the CPAN elasticsearch ?

It seems to list all CPAN releases with dates, links to tarballs and checksums. You could build a list of artifacts for each package as in the crates loader
and pass them as loader arguments.

Thanks for the review.

Yes I have checked this one too. When exploring CPAN I found a lot of inconsistencies when parsing data, so i've gone a simplier way.
The main cons is that /release/ returns a lot of useless data and a big amount of records, the idea is that the loader will load related versions via https://fastapi.metacpan.org/v1/release/versions/{pkgname}

Let's talk about this next week.

Sep 27 2022, 3:10 PM
anlambert requested review of D8550: developer-setup: Update packages list and repo URL.
Sep 27 2022, 2:55 PM
anlambert requested review of D8548: developer-setup: Add instructions to install latest yarn version.
Sep 27 2022, 2:15 PM
anlambert requested changes to D8171: crates: Loader implements incremental mode.

The loader requires some adaptation to align with lister output.

Sep 27 2022, 2:12 PM
anlambert added a comment to D8535: docker/conf: Declare task queues for new listers and loaders.

Are those new listers landed yet?

Not yet but they will be soon and it allows to test them in docker environment.

ok. If we land this now and someone starts from scratch or rebuild its local dev, that will break though.

Sep 27 2022, 2:08 PM
anlambert requested changes to D8454: Crates.io: Add last_update for each version of a crate.

@franckbret, I added the improvements for the crates lister we discussed last week as inline comments.

Sep 27 2022, 12:18 PM
anlambert closed D8537: browse: Add 'Save again' button in origin visits view.
Sep 27 2022, 11:50 AM
anlambert committed rDWAPPS4a03dc7bf6af: browse: Add 'Save again' button in origin visits view (authored by anlambert).
browse: Add 'Save again' button in origin visits view
Sep 27 2022, 11:50 AM
anlambert requested review of D8537: browse: Add 'Save again' button in origin visits view.
Sep 27 2022, 11:43 AM
anlambert accepted D8531: Puppet: Lister for Puppet modules.
Sep 27 2022, 11:23 AM
anlambert added a comment to D8542: Cpan: List Perl module origins from cpan.org.

@franckbret , have you considered exploiting the https://fastapi.metacpan.org/v1/release/_search endpoint of the CPAN elasticsearch ?

Sep 27 2022, 11:15 AM
anlambert added a comment to D8535: docker/conf: Declare task queues for new listers and loaders.

Are those new listers landed yet?

Sep 27 2022, 11:03 AM
anlambert accepted D8338: Hackage: List origins from hackage.haskell.org, The Haskell Package Repository.

Looks good to me !

Sep 27 2022, 10:47 AM
anlambert accepted D8517: Conda: List origins from anaconda.com, the Package, dependency and environment management for any language.

Looks good to me, thanks !

Sep 27 2022, 10:45 AM

Sep 26 2022

anlambert added a revision to T4553: Add “Save code now” to the Visits page: D8537: browse: Add 'Save again' button in origin visits view.
Sep 26 2022, 5:17 PM · Web app
anlambert requested changes to D8528: Nuget: Lister for NuGet the package manager for .NET.

Overall looks good to me, just a couple of inline comments to handle.

Sep 26 2022, 4:30 PM
anlambert requested changes to D8517: Conda: List origins from anaconda.com, the Package, dependency and environment management for any language.

Still some changes to bring to that diff now HTTP requests and user-agent setting codes have been deduplicated.

Sep 26 2022, 4:03 PM
anlambert requested review of D8536: Update instructions for running a lister in docker.
Sep 26 2022, 3:59 PM
anlambert requested review of D8535: docker/conf: Declare task queues for new listers and loaders.
Sep 26 2022, 3:39 PM
anlambert requested changes to D8379: Hackage: Loads Hackage Listed origins.

That diff requires some changes as the api_info function got renamed (see inline comments).

Sep 26 2022, 3:24 PM
anlambert requested changes to D8338: Hackage: List origins from hackage.haskell.org, The Haskell Package Repository.
Sep 26 2022, 2:50 PM
anlambert committed rDLDBASE26fe954bd9dc: golang: Fix imports ordering reported by isort (authored by anlambert).
golang: Fix imports ordering reported by isort
Sep 26 2022, 1:45 PM
anlambert added a comment to T4553: Add “Save code now” to the Visits page.

I think it is better to reuse the "Save again" button from code tab.

Sep 26 2022, 12:41 PM · Web app
anlambert triaged T4553: Add “Save code now” to the Visits page as Normal priority.
Sep 26 2022, 12:38 PM · Web app
anlambert triaged T4552: Add link to search for origins loaded from an add forge now request as Normal priority.
Sep 26 2022, 11:44 AM · Web app, Add Forge Now
anlambert closed D8523: Update value of User-Agent HTTP request header used by listers.
Sep 26 2022, 11:36 AM
anlambert closed D8520: Refactor and deduplicate HTTP requests code in listers.
Sep 26 2022, 11:36 AM
anlambert committed rDLSdb6ce12e9e3b: Refactor and deduplicate HTTP requests code in listers (authored by anlambert).
Refactor and deduplicate HTTP requests code in listers
Sep 26 2022, 11:36 AM
anlambert committed rDLSd5c30a3ce371: Update value of User-Agent HTTP request header used by listers (authored by anlambert).
Update value of User-Agent HTTP request header used by listers
Sep 26 2022, 11:36 AM
anlambert closed D8519: Use generic HTTP retry policy by default and rename dedicated decorator.
Sep 26 2022, 11:36 AM
anlambert committed rDLS9c55acd28609: Use generic HTTP retry policy by default and rename dedicated decorator (authored by anlambert).
Use generic HTTP retry policy by default and rename dedicated decorator
Sep 26 2022, 11:36 AM
anlambert updated the diff for D8523: Update value of User-Agent HTTP request header used by listers.

Rebase

Sep 26 2022, 10:51 AM
anlambert updated the diff for D8520: Refactor and deduplicate HTTP requests code in listers.

Rebase

Sep 26 2022, 10:50 AM
anlambert updated the diff for D8519: Use generic HTTP retry policy by default and rename dedicated decorator.

Rebase

Sep 26 2022, 10:50 AM
anlambert closed D8516: cypress: Use click command instead of submit one for form submission.
Sep 26 2022, 10:47 AM
anlambert committed rDWAPPS772928988c91: cypress: Use click command instead of submit one for form submission (authored by anlambert).
cypress: Use click command instead of submit one for form submission
Sep 26 2022, 10:47 AM

Sep 22 2022

anlambert requested review of D8523: Update value of User-Agent HTTP request header used by listers.
Sep 22 2022, 3:54 PM

Sep 21 2022

anlambert requested review of D8520: Refactor and deduplicate HTTP requests code in listers.
Sep 21 2022, 8:14 PM
anlambert requested review of D8519: Use generic HTTP retry policy by default and rename dedicated decorator.
Sep 21 2022, 5:07 PM
anlambert requested changes to D8517: Conda: List origins from anaconda.com, the Package, dependency and environment management for any language.

Overall, looks god to me but there is still some rooms for a couple of improvements (see inline comments).

Sep 21 2022, 4:17 PM
anlambert committed rDLDBASEe1f2135f241c: package/utils: rename api_info function to get_url_body (authored by anlambert).
package/utils: rename api_info function to get_url_body
Sep 21 2022, 3:49 PM
anlambert closed D8495: package/utils: Add debug log and throttling retry to api_info.
Sep 21 2022, 3:49 PM
anlambert committed rDLDBASEbcbb554e625f: package/utils: Add debug log and throttling retry to api_info (authored by anlambert).
package/utils: Add debug log and throttling retry to api_info
Sep 21 2022, 3:49 PM
anlambert updated the diff for D8495: package/utils: Add debug log and throttling retry to api_info.

Update doctring

Sep 21 2022, 3:45 PM
anlambert accepted D8515: Add filters for latest visit status.
Sep 21 2022, 3:27 PM
anlambert added a comment to T4478: Deploy Gogs lister to staging.

Nevertheless, they are less origins in the scheduler db, maybe some get listed more than once, need to check.

Sep 21 2022, 3:23 PM · Gogs lister, Origin-Gitea/Gogs, System administration
anlambert added inline comments to D8515: Add filters for latest visit status.
Sep 21 2022, 3:15 PM
anlambert accepted D8511: Change targettype in directory entry; add directory entry filter by name.

LGTM

Sep 21 2022, 2:57 PM
anlambert added a comment to T4478: Deploy Gogs lister to staging.

Just tested the lister in docker and I got a lot more origins listed:

docker-swh-lister-1  | [2022-09-21 12:24:57,583: INFO/MainProcess] Task swh.lister.gogs.tasks.FullGogsRelister[2b0ba017-4917-44dc-9511-e860bb431322] received
docker-swh-lister-1  | [2022-09-21 12:24:57,584: DEBUG/ForkPoolWorker-1] Loading config file /lister.yml
docker-swh-lister-1  | [2022-09-21 12:24:57,594: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search with params {'limit': 50, 'page': 1}
docker-swh-lister-1  | [2022-09-21 12:24:58,556: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=2 with params {}
docker-swh-lister-1  | [2022-09-21 12:24:58,791: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=3 with params {}
docker-swh-lister-1  | [2022-09-21 12:24:59,018: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=4 with params {}
docker-swh-lister-1  | [2022-09-21 12:24:59,263: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=5 with params {}
docker-swh-lister-1  | [2022-09-21 12:24:59,580: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=6 with params {}
docker-swh-lister-1  | [2022-09-21 12:24:59,845: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=7 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:00,094: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=8 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:00,323: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=9 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:00,560: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=10 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:00,787: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=11 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:01,020: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=12 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:01,324: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=13 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:01,794: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=14 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:02,191: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=15 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:02,549: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=16 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:02,967: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=17 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:03,250: WARNING/ForkPoolWorker-1] Unexpected HTTP status code 500 on https://try.gogs.io/api/v1/repos/search?page=17: b''
docker-swh-lister-1  | [2022-09-21 12:25:03,250: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=18 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:03,515: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=19 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:03,765: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=20 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:04,078: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=21 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:04,444: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=22 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:04,695: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=23 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:04,948: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=24 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:05,262: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=25 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:05,814: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=26 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:06,184: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=27 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:06,454: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=28 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:06,688: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=29 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:06,935: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=30 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:07,247: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=31 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:07,500: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=32 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:07,734: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=33 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:08,064: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=34 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:08,337: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=35 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:08,581: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=36 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:08,818: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=37 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:09,057: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=38 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:09,289: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=39 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:09,531: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=40 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:09,756: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=41 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:09,999: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=42 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:10,213: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=43 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:10,444: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=44 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:10,689: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=45 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:10,914: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=46 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:11,163: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=47 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:11,390: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=48 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:11,710: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=49 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:12,068: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=50 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:12,342: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=51 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:12,619: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=52 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:12,862: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=53 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:13,169: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=54 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:13,549: DEBUG/ForkPoolWorker-1] Fetching URL https://try.gogs.io/api/v1/repos/search?page=55 with params {}
docker-swh-lister-1  | [2022-09-21 12:25:13,827: DEBUG/ForkPoolWorker-1] Start from server, version: 0.9, properties: {'capabilities': {'publisher_confirms': True, 'exchange_exchange_bindings': True, 'basic.nack': True, 'consumer_cancel_notify': True, 'connection.blocked': True, 'consumer_priorities': True, 'authentication_failure_close': True, 'per_consumer_qos': True, 'direct_reply_to': True}, 'cluster_name': 'rabbit@0f4428ad1388', 'copyright': 'Copyright (C) 2007-2018 Pivotal Software, Inc.', 'information': 'Licensed under the MPL.  See http://www.rabbitmq.com/', 'platform': 'Erlang/OTP 19.2.1', 'product': 'RabbitMQ', 'version': '3.6.16'}, mechanisms: [b'AMQPLAIN', b'PLAIN'], locales: ['en_US']
docker-swh-lister-1  | [2022-09-21 12:25:13,828: DEBUG/ForkPoolWorker-1] using channel_id: 1
docker-swh-lister-1  | [2022-09-21 12:25:13,831: DEBUG/ForkPoolWorker-1] Channel open
docker-swh-lister-1  | [2022-09-21 12:25:13,833: INFO/ForkPoolWorker-1] Task swh.lister.gogs.tasks.FullGogsRelister[2b0ba017-4917-44dc-9511-e860bb431322] succeeded in 16.23296177299926s: {'pages': 55, 'origins': 575}
Sep 21 2022, 2:33 PM · Gogs lister, Origin-Gitea/Gogs, System administration
anlambert added a comment to T4478: Deploy Gogs lister to staging.

I was wondering why the number of listed repos was so low, turns out gogs has an option to not list a repository through the API when creating it:

Sep 21 2022, 2:23 PM · Gogs lister, Origin-Gitea/Gogs, System administration
anlambert requested changes to D8511: Change targettype in directory entry; add directory entry filter by name.

Using .lower() on byte strings of unknown encoding is going to have surprising results.

What is the intended use of making this case-insensitive?

The intention is to have some kind of a search for contents. Do you think a case sensitive filter is good enough, or should we explicitly handle the error here?

Sep 21 2022, 2:09 PM
anlambert requested changes to D8515: Add filters for latest visit status.
Sep 21 2022, 1:53 PM
anlambert requested review of D8516: cypress: Use click command instead of submit one for form submission.
Sep 21 2022, 1:51 PM
anlambert accepted D8513: use 'directorySwhid' instead of 'swhid' while querying a DirectoryEntry.
Sep 21 2022, 1:43 PM
anlambert added a comment to D8514: Fix validateForgeUrl.

Argh, why cypress tests did not catch this ? thanks !

Sep 21 2022, 11:10 AM
anlambert added a project to T4547: Create a Conda Lister: Conda lister.
Sep 21 2022, 10:59 AM · Conda lister
anlambert created Conda lister.
Sep 21 2022, 10:58 AM
anlambert accepted D8514: Fix validateForgeUrl.

Argh, why cypress tests did not catch this ? thanks !

Sep 21 2022, 10:52 AM

Sep 20 2022

anlambert accepted D8508: cgit: Ensure the clone url is searched on the right tab.

Looks good to me, thanks !

Sep 20 2022, 4:48 PM
anlambert updated the diff for D8495: package/utils: Add debug log and throttling retry to api_info.

Rebase

Sep 20 2022, 2:58 PM
anlambert closed D8504: package/loader: Implement load_status and visit_status.
Sep 20 2022, 12:10 PM
anlambert closed D8505: package/loader: Handle errors when retrieving package version info.
Sep 20 2022, 12:10 PM
anlambert committed rDLDBASEd471cca9f2a3: package/loader: Handle errors when retrieving package version info (authored by anlambert).
package/loader: Handle errors when retrieving package version info
Sep 20 2022, 12:10 PM
anlambert committed rDLDBASE598873f00392: package/loader: Implement load_status and visit_status (authored by anlambert).
package/loader: Implement load_status and visit_status
Sep 20 2022, 12:10 PM
anlambert closed D8503: package/tests: Add a visit success test for StubPackageLoader.
Sep 20 2022, 12:10 PM
anlambert committed rDLDBASE63ccbd9a3a67: package/tests: Add a visit success test for StubPackageLoader (authored by anlambert).
package/tests: Add a visit success test for StubPackageLoader
Sep 20 2022, 12:10 PM
anlambert updated the diff for D8505: package/loader: Handle errors when retrieving package version info.

Rebase

Sep 20 2022, 12:05 PM
anlambert updated the diff for D8504: package/loader: Implement load_status and visit_status.

Rebase

Sep 20 2022, 12:04 PM
anlambert updated the diff for D8503: package/tests: Add a visit success test for StubPackageLoader.

Rebase and improve test

Sep 20 2022, 12:04 PM
anlambert added inline comments to D8503: package/tests: Add a visit success test for StubPackageLoader.
Sep 20 2022, 12:01 PM
anlambert closed D8506: golang: Ensure to include dev version when no releases exist.
Sep 20 2022, 11:58 AM
anlambert committed rDLDBASEc4370009d0a5: golang: Ensure to include dev version when no releases exist (authored by anlambert).
golang: Ensure to include dev version when no releases exist
Sep 20 2022, 11:58 AM
anlambert requested changes to D8508: cgit: Ensure the clone url is searched on the right tab.
Sep 20 2022, 11:28 AM

Sep 19 2022

anlambert requested review of D8506: golang: Ensure to include dev version when no releases exist.
Sep 19 2022, 5:13 PM
anlambert added a revision to T4124: Golang support: D8506: golang: Ensure to include dev version when no releases exist.
Sep 19 2022, 5:10 PM · Golang loader, Golang lister, Archive coverage
anlambert requested review of D8505: package/loader: Handle errors when retrieving package version info.
Sep 19 2022, 4:49 PM