Yes, thanks for the update.

Build has FAILED

Fix ardumont comments

And first, fix the model appropriately...
for gnu, the uid (primary key) should at least be the url... the name 3dldf for example exists at least for gnu and old-gnu entries so that cannot work.

My feeling about this is that the lister should flush more often and deal more appropriately with unicity constraint failure.
Prior to removing altogether the db i mean.

Mmm, it's not the debian lister specific it seems...

Please, see my latest comments.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/491/ for more details.

• Move sources.json mock to correct location

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/778/ for more details.

Build has FAILED

• Rename JSONLister to FunctionalPackageLister
• Fix test
• Cleaning

I guess it's time to get rid of this database? 0:-)

Build has FAILED

• wip: switch to the new format

LGTM, though I think we should define the git type right away. For that, we can probably reuse a format similar to that found at https://guix.gnu.org/packages.json, which looks like:

{
    "type": "git",
    "git_url": "https://github.com/pali/0xffff.git",
    "git_ref": "0.8"
}

... where git_ref can be a tag name or a commit ID.

WDYT?

Hi @lewo and all!

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/468/ for more details.

Fix type annotations and squash some commits

Build has FAILED

Rebase and plug to latest master

I discussed with Ludo and we agreed on the fact the current packages.json file is not really suitable for the SWH usecase.
tl;dr the idea is to expose a list of sources instead of a list of packages.

Since Ludo (@civodul) posted this WIP feature on guix-devel mailing list [1] I am trying to follow this thread and I would like to help.

Since Ludo (@civodul) posted this WIP feature on guix-devel mailing list [1] I am trying to follow this thread and I would like to help.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/461/ for more details.

Rebase to latest master

Also ci job currently fails for the pep8 violations [1]:

I think my branch is almost working (i still need to rename the lister).

Build has FAILED

• [wip] swh.lister.json: Add lister getting sources from JSON file
• Rebased

By the way, i entertained the idea to write a guix lister with the current api, would that help (that could demo the tests to write as well)?

Build was aborted

Build was aborted

• gnu.tree: Use .format build

Build was aborted

Build was aborted

Adapt according to what's possible from the review

In D2147#50755, @ardumont wrote:

I don't think this regexp is the right solution, it's quite fragile in case a package name uses one of the keywords.

Yes, it's fragile and we don't see any better solution right now.
That's how we did it initially, we improved the current version to be a tad better though.
And also it's tested even if it's limited.

A possible solution is for get_version to only accept URLs, and split the path to find the package name.

I'm not getting the difference with the current implem.
filename is extracted from the uri in any case...

I don't think this regexp is the right solution, it's quite fragile in case a package name uses one of the keywords.

I don't think this regexp is the right solution, it's quite fragile in case a package name uses one of the keywords. A possible solution is for get_version to only accept URLs, and split the path to find the package name.

Build has FAILED

Build has FAILED

Fix parsing timestamps

Build has FAILED

Build has FAILED

• gnu.lister: Format timestamp to isoformat string for the tar loader
• gnu.lister: Unify timestamp formats to isoformat date in model
• gnu.tree: Drop debug statement
• gnu.lister: Update docstring samples

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/446/ for more details.

Rebase to latest master

@ardumont @douardda Regarding the lister name, I agree the JSONLister name is not appropriate. @ardumont proposed FunctionalPackageManagerLister or FunctionalPackageLister but I'm not sure they are appropriated too :)

@ardumont @douardda Regarding the lister name, I agree the JSONLister name is not appropriate. @ardumont proposed FunctionalPackageManagerLister or FunctionalPackageLister but I'm not sure they are appropriated too :)

In D2025#50269, @douardda wrote:

Just one note: the name is IMHO inappropriate. This is NOT a JSON lister. JSON is nothing but the serialization format used to retrieve some (more or less) well defined structured data.

What defines this lister is its ability to comprehend the very data structure mentioned in this diff's description.

Indeed, i proposed a name in D2025#inline-13301 ;)

Just one note: the name is IMHO inappropriate. This is NOT a JSON lister. JSON is nothing but the serialization format used to retrieve some (more or less) well defined structured data.