Page MenuHomeSoftware Heritage

ListerFolder
ActivePublic

Members

  • This project does not have any members.

Recent Activity

Fri, Jul 19

ardumont closed T1923: Implement Packagist Lister, a subtask of T1776: packagist (PHP) Lister, as Resolved.
Fri, Jul 19, 4:48 PM · Lister, Archive coverage
ardumont closed T1923: Implement Packagist Lister as Resolved.
Fri, Jul 19, 4:48 PM · Lister, Archive coverage
ardumont added a revision to T1923: Implement Packagist Lister: D1584: swh.lister.packagist.
Fri, Jul 19, 4:36 PM · Lister, Archive coverage
ardumont triaged T1924: Deploy packagist Lister as Normal priority.
Fri, Jul 19, 4:36 PM · Lister, Archive coverage
ardumont triaged T1923: Implement Packagist Lister as Normal priority.
Fri, Jul 19, 4:35 PM · Lister, Archive coverage

Thu, Jul 18

nahimilega closed T1890: pypi lister: Add tests as Resolved by committing rDLS08ade29e6de0: swh.lister.pypi: Add tests.
Thu, Jul 18, 6:50 PM · Origin-Pypi, Lister

Sun, Jul 14

nahimilega added a revision to T1890: pypi lister: Add tests: D1733: swh.lister.core: Add tests for simple lister.
Sun, Jul 14, 8:28 PM · Origin-Pypi, Lister

Wed, Jul 10

ardumont added a comment to T1906: list-gitlab-full on gitlab.com crashes with: TypeError: 'NoneType' object cannot be interpreted as an integer.

I think it's related to T1863.

Wed, Jul 10, 10:19 AM · Origin-GitLab, Lister
ardumont updated the task description for T1863: gitlab.com: Full lister fails to retrieve the range information it needs to start listing.
Wed, Jul 10, 10:17 AM · Lister
vlorentz triaged T1906: list-gitlab-full on gitlab.com crashes with: TypeError: 'NoneType' object cannot be interpreted as an integer as Normal priority.
Wed, Jul 10, 10:16 AM · Origin-GitLab, Lister
vlorentz updated the task description for T1906: list-gitlab-full on gitlab.com crashes with: TypeError: 'NoneType' object cannot be interpreted as an integer.
Wed, Jul 10, 10:13 AM · Origin-GitLab, Lister
vlorentz created T1906: list-gitlab-full on gitlab.com crashes with: TypeError: 'NoneType' object cannot be interpreted as an integer.
Wed, Jul 10, 10:12 AM · Origin-GitLab, Lister

Mon, Jul 8

ardumont triaged T1890: pypi lister: Add tests as Normal priority.
Mon, Jul 8, 10:29 AM · Origin-Pypi, Lister

Thu, Jul 4

ardumont updated the title for P459 ugly unstucking bitbucket lister from irk: unstuck bitbucket lister to ugly unstucking bitbucket lister.
Thu, Jul 4, 4:58 PM · Origin-Bitbucket, Bitbucket lister, Lister
ardumont added projects to P459 ugly unstucking bitbucket lister: Lister, Bitbucket lister, Origin-Bitbucket.
Thu, Jul 4, 4:57 PM · Origin-Bitbucket, Bitbucket lister, Lister
ardumont updated the task description for T1859: bitbucket lister: Deal with connection errors.
Thu, Jul 4, 4:36 PM · Lister, Bitbucket lister
ardumont added a comment to T1859: bitbucket lister: Deal with connection errors.

I have cc-ed swh-devel@inria.fr, no idea if that will work or not though

Thu, Jul 4, 4:31 PM · Lister, Bitbucket lister
ardumont changed the status of T1859: bitbucket lister: Deal with connection errors from Open to Work in Progress.
Thu, Jul 4, 11:30 AM · Lister, Bitbucket lister
ardumont added a comment to T1859: bitbucket lister: Deal with connection errors.

I have reported the issue to the atlassian team [1] with mostly the following analysis:

Thu, Jul 4, 11:30 AM · Lister, Bitbucket lister
ardumont added a comment to T1859: bitbucket lister: Deal with connection errors.

So possibly a blacklist per range of ips.

Thu, Jul 4, 10:49 AM · Lister, Bitbucket lister
ardumont updated the task description for T1859: bitbucket lister: Deal with connection errors.
Thu, Jul 4, 10:01 AM · Lister, Bitbucket lister
ardumont updated the task description for T1859: bitbucket lister: Deal with connection errors.
Thu, Jul 4, 10:01 AM · Lister, Bitbucket lister

Wed, Jul 3

ardumont placed T1863: gitlab.com: Full lister fails to retrieve the range information it needs to start listing up for grabs.
Wed, Jul 3, 3:26 PM · Lister

Mon, Jul 1

ardumont renamed T1835: List/Ingest major cgit instances from List major cgit instances to List/Ingest major cgit instances.
Mon, Jul 1, 4:52 PM · Lister
ardumont added a comment to T1835: List/Ingest major cgit instances.

Getting gnu-savannah instance out of the task (T1451 exists for it already).

Mon, Jul 1, 4:51 PM · Lister
ardumont updated the task description for T1835: List/Ingest major cgit instances.
Mon, Jul 1, 4:04 PM · Lister
ardumont updated the task description for T1835: List/Ingest major cgit instances.
Mon, Jul 1, 4:04 PM · Lister
ardumont updated the task description for T1835: List/Ingest major cgit instances.
Mon, Jul 1, 4:04 PM · Lister
ardumont updated the task description for T1835: List/Ingest major cgit instances.
Mon, Jul 1, 4:03 PM · Lister
ardumont updated the task description for T1835: List/Ingest major cgit instances.
Mon, Jul 1, 4:01 PM · Lister
ardumont renamed T1865: gitlab lister: make full listing on large instance more robust to concurrency writings from gitlab lister: full listing on large instance does not handle correctly concurrency writings to gitlab lister: make full listing on large instance more robust to concurrency writings.
Mon, Jul 1, 10:16 AM · Lister
ardumont updated the task description for T1866: gitlab-lister: Investigate rate limit implementation further.
Mon, Jul 1, 10:04 AM · Lister

Sun, Jun 30

ardumont added a comment to T1866: gitlab-lister: Investigate rate limit implementation further.

I did not find anything specific to rate limit policy for salsa.
I found an api usage best practices chapter though [1].

Sun, Jun 30, 6:50 PM · Lister
ardumont triaged T1866: gitlab-lister: Investigate rate limit implementation further as Normal priority.
Sun, Jun 30, 11:55 AM · Lister
ardumont added a project to T1865: gitlab lister: make full listing on large instance more robust to concurrency writings: Lister.
Sun, Jun 30, 11:08 AM · Lister
ardumont renamed T1863: gitlab.com: Full lister fails to retrieve the range information it needs to start listing from gitlab lister: Full lister fails to retrieve the range information it needs to start listing to gitlab.com: Full lister fails to retrieve the range information it needs to start listing.
Sun, Jun 30, 11:00 AM · Lister
ardumont added a comment to T1863: gitlab.com: Full lister fails to retrieve the range information it needs to start listing.

I'll just concentrate on fixing the urls and respawn those tasks for now.

Sun, Jun 30, 10:56 AM · Lister
ardumont changed the status of T1863: gitlab.com: Full lister fails to retrieve the range information it needs to start listing from Open to Work in Progress.

Heads up.

Sun, Jun 30, 10:47 AM · Lister

Sat, Jun 29

ardumont updated the task description for T1861: cgit lister: Adapt lister to deal with inconsistent `git clone uri` pattern.
Sat, Jun 29, 3:57 PM · CGit lister, Lister
ardumont updated the task description for T1863: gitlab.com: Full lister fails to retrieve the range information it needs to start listing.
Sat, Jun 29, 9:54 AM · Lister
ardumont triaged T1863: gitlab.com: Full lister fails to retrieve the range information it needs to start listing as High priority.
Sat, Jun 29, 9:53 AM · Lister
ardumont updated the task description for T1861: cgit lister: Adapt lister to deal with inconsistent `git clone uri` pattern.
Sat, Jun 29, 9:15 AM · CGit lister, Lister
ardumont added a comment to T1861: cgit lister: Adapt lister to deal with inconsistent `git clone uri` pattern.

I'd be inclined to use a composition of solution:

  • having the listing policy determined at cgit instance lister initialization (3.)
  • Since 1. has already been done in the past, use that instead as a fallback (eclipse and freedesktop might be popular enough to sustain the load of a bit more request).
Sat, Jun 29, 9:13 AM · CGit lister, Lister
ardumont renamed T1861: cgit lister: Adapt lister to deal with inconsistent `git clone uri` pattern from cgit lister adaptations to deal with cgit.freedesktop.org to cgit lister: Adapt lister to deal with inconsistent `git clone uri` pattern.
Sat, Jun 29, 9:10 AM · CGit lister, Lister

Fri, Jun 28

ardumont added a comment to T1800: gitweb lister.

Related T1659?

It's not really related, because gitweb and cgit are two different things.

Fri, Jun 28, 8:48 PM · Lister
ardumont updated the task description for T1861: cgit lister: Adapt lister to deal with inconsistent `git clone uri` pattern.
Fri, Jun 28, 8:36 PM · CGit lister, Lister
ardumont added a comment to T1835: List/Ingest major cgit instances.

Getting cgit.freedesktop.org out of the scope of this task.
See T1861 for it.

Fri, Jun 28, 8:14 PM · Lister
ardumont updated the task description for T1835: List/Ingest major cgit instances.
Fri, Jun 28, 8:14 PM · Lister
ardumont removed a subtask for T1835: List/Ingest major cgit instances: T1451: ingest GNU Savannah Git repositories.
Fri, Jun 28, 8:12 PM · Lister
ardumont closed T1835: List/Ingest major cgit instances as Resolved.
Fri, Jun 28, 8:12 PM · Lister