Page MenuHomeSoftware Heritage

ListerFolder
ActivePublic

Members

  • This project does not have any members.
  • View All

Recent Activity

Today

ardumont updated the task description for T1991: Implement a Guix/Nix loader.
Thu, Apr 2, 4:34 PM · Core Loader, Lister

Tue, Mar 24

legau added a comment to T1734: Create a Lister for launchpad.net.

Hi, I proposed a first version with the new changes (D2799).
@cjwatson it should be coherent with your snippet.
If somebody is used to be working with listers I would be glad to hear some remarks over how I implemented it.

Tue, Mar 24, 3:50 PM · Lister, Archive coverage
zack updated the task description for T2313: Archive git.fsfe.org (Gitea).
Tue, Mar 24, 1:40 PM · Archive coverage, Lister

Thu, Mar 19

lewo updated the task description for T1991: Implement a Guix/Nix loader.
Thu, Mar 19, 11:09 AM · Core Loader, Lister

Wed, Mar 18

ardumont added a comment to T1991: Implement a Guix/Nix loader.

First runs triggered on staging, errors will be reported in sentry [1]

Wed, Mar 18, 4:50 PM · Core Loader, Lister

Fri, Mar 13

olasd edited projects for T1721: Implementation of Gogs Lister, added: Lister; removed Archive coverage.
Fri, Mar 13, 1:27 PM · Lister
olasd added a comment to T2313: Archive git.fsfe.org (Gitea).

Unfortunately, try.gogs.io's API is hidden behind auth so I can't confirm that the responses actually have the same shape between gogs and gitea.

Fri, Mar 13, 1:26 PM · Archive coverage, Lister
olasd added a comment to T2313: Archive git.fsfe.org (Gitea).

Thanks for submitting this request. There's a good chance that this can be the same lister as gogs: T1721.

Fri, Mar 13, 1:25 PM · Archive coverage, Lister
olasd added projects to T2313: Archive git.fsfe.org (Gitea): Lister, Archive coverage.
Fri, Mar 13, 1:21 PM · Archive coverage, Lister
lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

@civodul @zimoun I'm wondering if you generate a sources.json file for any commit of your guix repository or only for those that have been evaluated and pushed to your binary cache by your CI?

Fri, Mar 13, 10:36 AM · Lister

Thu, Mar 12

cjwatson added a comment to T1734: Create a Lister for launchpad.net.

Yep, I'm not annoyed, just being emphatic about what we want to see. :-)

Thu, Mar 12, 6:29 PM · Lister, Archive coverage
zack added a comment to T1734: Create a Lister for launchpad.net.

Hi Colin (@cjwatson), nice to meet you here !

Thu, Mar 12, 1:35 PM · Lister, Archive coverage

Wed, Mar 11

cjwatson added a comment to T1734: Create a Lister for launchpad.net.

I'm one of the developers on the Launchpad team. A user identified as "leni" spoke to us about this on IRC last week; it so happened that the Launchpad team were in the middle of an in-person sprint at the time, so we were able to discuss the problem fairly quickly and put together a plan to improve our API. I implemented those improvements shortly afterwards. They aren't quite deployed on production yet, but they should be very soon. Unfortunately I don't have any contact details for leni unless they happen to join IRC, so I'm posting a summary of the discussion and my improvements here, which is probably a useful thing to do anyway.

Wed, Mar 11, 11:09 PM · Lister, Archive coverage

Tue, Mar 10

lewo updated the task description for T1991: Implement a Guix/Nix loader.
Tue, Mar 10, 6:49 PM · Core Loader, Lister
lewo updated the task description for T1991: Implement a Guix/Nix loader.
Tue, Mar 10, 6:42 PM · Core Loader, Lister

Mon, Mar 9

lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

Thanks to @zimoun, https://guix.gnu.org/sources.json is now generated periodically (every hour). Each url is now a list.

Mon, Mar 9, 2:35 PM · Lister
civodul added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
In D2025#65063, @lewo wrote:

While looking into this with @zimoun, we realized it would be nicer if url were an array of URLs (as is the case at https://guix.gnu.org/packages.json) rather than a single URL.

without changing now the crawler, i.e., the crawler can ingest only the first elem of the array and it will be modified later.

Yeah, I was thinking to introduce this later. But as you said, we could still modify the format without supporting it in the lister.
So, that's fine for me if we generate a list of urls instead of a single url. I could easily update the file NixOS is generating.

Mon, Mar 9, 1:52 PM · Lister

Fri, Mar 6

lewo abandoned D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

@lewo : Let me know the new diff number. :-)

Fri, Mar 6, 5:40 PM · Lister
zimoun added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

After some discussions with the SWH team, it is actually no longer the good way to fill the archive with our sources. Instead, I'm starting to write a loader which will be in charge of reading our sources.json and fill the archive. So, I'm closing this diff and will create a new diff with a loader in the next few days;)

Fri, Mar 6, 10:14 AM · Lister

Thu, Mar 5

ardumont assigned T1991: Implement a Guix/Nix loader to lewo.
Thu, Mar 5, 7:30 PM · Core Loader, Lister
ardumont renamed T1991: Implement a Guix/Nix loader from Implement a Guix/GuixSD/Nix/NixOS loader to Implement a Guix/Nix loader.
Thu, Mar 5, 7:30 PM · Core Loader, Lister
ardumont added a comment to T1991: Implement a Guix/Nix loader.

Implementation of a lister started at D2025.

Thu, Mar 5, 7:30 PM · Core Loader, Lister
ardumont renamed T1991: Implement a Guix/Nix loader from Implement a Guix/GuixSD/Nix/NixOS lister to Implement a Guix/GuixSD/Nix/NixOS loader.
Thu, Mar 5, 7:28 PM · Core Loader, Lister
lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

After some discussions with the SWH team, it is actually no longer the good way to fill the archive with our sources. Instead, I'm starting to write a loader which will be in charge of reading our sources.json and fill the archive. So, I'm closing this diff and will create a new diff with a loader in the next few days;)
There are also some advantages of implementing a loader: for instance, we could query the SWH API to know which sources of a specific sources.json file have been archived!

Thu, Mar 5, 4:36 PM · Lister

Mar 2 2020

zimoun added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

@lewo: Does the version of the format should be bumped to 2 with this string-to-array modification?

No, I don't think so since it is not used yet.

Mar 2 2020, 7:57 PM · Lister
lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

> @lewo: Does the version of the format should be bumped to 2 with this string-to-array modification?

Mar 2 2020, 6:52 PM · Lister
zimoun added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

@lewo: Does the version of the format should be bumped to 2 with this string-to-array modification?

Mar 2 2020, 5:26 PM · Lister

Feb 27 2020

lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

While looking into this with @zimoun, we realized it would be nicer if url were an array of URLs (as is the case at https://guix.gnu.org/packages.json) rather than a single URL.

without changing now the crawler, i.e., the crawler can ingest only the first elem of the array and it will be modified later.

Feb 27 2020, 10:24 PM · Lister

Feb 18 2020

zimoun added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

While looking into this with @zimoun, we realized it would be nicer if url were an array of URLs (as is the case at https://guix.gnu.org/packages.json) rather than a single URL.

The reason is that in many cases, both Guix and Nixpkgs provide a list of URLs rather than a single URL, which is useful when one of them breaks.

Feb 18 2020, 10:01 PM · Lister
civodul added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
In D2025#61931, @lewo wrote:

A CI job is building a sources.json every day! The file is available at https://nix-community.github.io/nixpkgs-swh/sources.json ;)

Awesome!

Feb 18 2020, 9:23 AM · Lister

Feb 14 2020

legau moved T2260: Fully annotate swh-lister with types from Backlog to In progress on the Easy hack board.
Feb 14 2020, 2:39 PM · Easy hack, Lister

Feb 13 2020

legau added a comment to T1734: Create a Lister for launchpad.net.

What is the current status of this task ?

Feb 13 2020, 2:34 PM · Lister, Archive coverage

Jan 30 2020

lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
> If you are going to the FOSDEM, would be nice to meet you there to talk about next steps!

It would but i'm not going.
Jan 30 2020, 8:56 PM · Lister
lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
> If you are going to the FOSDEM, would be nice to meet you there to talk about next steps!

I'm already in Brussels and would be happy to meet!
Jan 30 2020, 8:53 PM · Lister
ardumont updated subscribers of D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

A CI job is building a sources.json every day! The file is available at https://nix-community.github.io/nixpkgs-swh/sources.json ;)

This is a community CI (not hosted on main NixOS infrascture) which will allow me to iterate quickly on this file.

Jan 30 2020, 10:30 AM · Lister
civodul added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
In D2025#61931, @lewo wrote:

A CI job is building a sources.json every day! The file is available at https://nix-community.github.io/nixpkgs-swh/sources.json ;)

Jan 30 2020, 12:44 AM · Lister

Jan 29 2020

lewo added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

A CI job is building a sources.json every day! The file is available at https://nix-community.github.io/nixpkgs-swh/sources.json ;)
This is a community CI (not hosted on main NixOS infrascture) which will allow me to iterate quickly on this file.

Jan 29 2020, 11:39 PM · Lister
lewo updated the summary of D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
Jan 29 2020, 11:33 PM · Lister
vlorentz added a project to T2260: Fully annotate swh-lister with types: Easy hack.
Jan 29 2020, 3:38 PM · Easy hack, Lister
vlorentz updated the task description for T2260: Fully annotate swh-lister with types.
Jan 29 2020, 3:38 PM · Easy hack, Lister
vlorentz triaged T2260: Fully annotate swh-lister with types as Low priority.
Jan 29 2020, 3:37 PM · Easy hack, Lister

Jan 27 2020

vlorentz added a project to T1734: Create a Lister for launchpad.net: Lister.
Jan 27 2020, 4:31 PM · Lister, Archive coverage

Jan 22 2020

vlorentz added a project to T2200: Automatic forge discovery: Lister.
Jan 22 2020, 4:41 PM · Lister, Restricted Project
ardumont renamed T2241: lister/loader: Ingest archived artifacts from cran mirror from lister/loader: Ingest all known artifacts from cran mirror to lister/loader: Ingest archived artifacts from cran mirror.
Jan 22 2020, 10:46 AM · Core Loader, Lister, Origin-CRAN

Jan 21 2020

lewo retitled D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file from [wip] swh.lister.json: Add lister getting sources from JSON file to [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
Jan 21 2020, 7:12 PM · Lister
ardumont added a comment to T2241: lister/loader: Ingest archived artifacts from cran mirror.

This should take care of [1]

Jan 21 2020, 12:18 PM · Core Loader, Lister, Origin-CRAN
ardumont updated the task description for T2241: lister/loader: Ingest archived artifacts from cran mirror.
Jan 21 2020, 11:52 AM · Core Loader, Lister, Origin-CRAN
ardumont triaged T2241: lister/loader: Ingest archived artifacts from cran mirror as Normal priority.
Jan 21 2020, 11:52 AM · Core Loader, Lister, Origin-CRAN

Jan 17 2020

ardumont added a comment to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.

Could you please update the title and the description according to the current state?
(i you don't have time, please tell me so i will ;)

Jan 17 2020, 9:17 AM · Lister
ardumont added inline comments to D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
Jan 17 2020, 8:41 AM · Lister