Page MenuHomeSoftware Heritage
Feed Advanced Search

Thu, Nov 21

ardumont added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

See console output for more information: https://jenkins.softwareheritage.org/job/DLS/job/tox/494/console

Thu, Nov 21, 2:26 PM · Lister
lewo added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Thu, Nov 21, 12:16 AM · Lister
Harbormaster failed remote builds in B9180: Diff 7968 for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file!
Thu, Nov 21, 12:15 AM · Lister
swh-public-ci added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Build has FAILED

Thu, Nov 21, 12:15 AM · Lister
lewo updated the diff for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Fix ardumont comments

Thu, Nov 21, 12:12 AM · Lister

Fri, Nov 15

ardumont added a comment to T2070: lister: Failure during listing on a fresh new db (unicity constraint).

And first, fix the model appropriately...
for gnu, the uid (primary key) should at least be the url... the name 3dldf for example exists at least for gnu and old-gnu entries so that cannot work.

Fri, Nov 15, 11:23 AM · Lister
ardumont added a comment to T2070: lister: Failure during listing on a fresh new db (unicity constraint).

My feeling about this is that the lister should flush more often and deal more appropriately with unicity constraint failure.
Prior to removing altogether the db i mean.

Fri, Nov 15, 11:18 AM · Lister
ardumont renamed T2070: lister: Failure during listing on a fresh new db (unicity constraint) from lister-debian: Failure during listing on a fresh new db (unicity constraint) to lister: Failure during listing on a fresh new db (unicity constraint).
Fri, Nov 15, 11:17 AM · Lister
ardumont added a comment to T2070: lister: Failure during listing on a fresh new db (unicity constraint).

Mmm, it's not the debian lister specific it seems...

Fri, Nov 15, 11:17 AM · Lister
ardumont added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Please, see my latest comments.

Fri, Nov 15, 10:41 AM · Lister

Thu, Nov 14

swh-public-ci added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/491/ for more details.

Thu, Nov 14, 8:23 PM · Lister
lewo added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Thu, Nov 14, 8:22 PM · Lister
lewo updated the diff for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
  • Move sources.json mock to correct location
Thu, Nov 14, 8:19 PM · Lister
ardumont closed D2272: swh.storage.schemata: Drop schemata from storage.
Thu, Nov 14, 1:31 PM · Storage manager, Lister
olasd accepted D2272: swh.storage.schemata: Drop schemata from storage.
Thu, Nov 14, 12:12 PM · Storage manager, Lister
ardumont closed T2032: listers: Add proper integration tests to lister as Resolved.
Thu, Nov 14, 10:40 AM · Lister
swh-public-ci added a comment to D2272: swh.storage.schemata: Drop schemata from storage.

Build is green
See https://jenkins.softwareheritage.org/job/DSTO/job/tox/778/ for more details.

Thu, Nov 14, 10:40 AM · Storage manager, Lister
ardumont updated the summary of D2272: swh.storage.schemata: Drop schemata from storage.
Thu, Nov 14, 10:39 AM · Storage manager, Lister

Wed, Nov 13

ardumont added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Wed, Nov 13, 5:06 PM · Lister
Harbormaster failed remote builds in B8979: Diff 7800 for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file!
Wed, Nov 13, 9:46 AM · Lister
swh-public-ci added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Build has FAILED

Wed, Nov 13, 9:46 AM · Lister
lewo updated the diff for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
  • Rename JSONLister to FunctionalPackageLister
  • Fix test
  • Cleaning
Wed, Nov 13, 9:43 AM · Lister

Fri, Nov 8

olasd added a comment to T2070: lister: Failure during listing on a fresh new db (unicity constraint).

I guess it's time to get rid of this database? 0:-)

Fri, Nov 8, 1:38 PM · Lister
ardumont renamed T2070: lister: Failure during listing on a fresh new db (unicity constraint) from lister-debian: Failure during listing (unicity constraint) to lister-debian: Failure during listing on a fresh new db (unicity constraint).
Fri, Nov 8, 12:18 PM · Lister
ardumont triaged T2070: lister: Failure during listing on a fresh new db (unicity constraint) as Normal priority.
Fri, Nov 8, 11:58 AM · Lister

Nov 6 2019

Harbormaster failed remote builds in B8851: Diff 7683 for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file!
Nov 6 2019, 9:55 PM · Lister
swh-public-ci added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Build has FAILED

Nov 6 2019, 9:55 PM · Lister
lewo updated the diff for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
  • wip: switch to the new format
Nov 6 2019, 9:54 PM · Lister
lewo added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

LGTM, though I think we should define the git type right away. For that, we can probably reuse a format similar to that found at https://guix.gnu.org/packages.json, which looks like:

{
    "type": "git",
    "git_url": "https://github.com/pali/0xffff.git",
    "git_ref": "0.8"
}

... where git_ref can be a tag name or a commit ID.
WDYT?

Nov 6 2019, 9:52 PM · Lister

Nov 5 2019

civodul added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Hi @lewo and all!

Nov 5 2019, 3:40 PM · Lister

Nov 4 2019

ardumont closed D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Nov 4 2019, 11:13 AM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/468/ for more details.

Nov 4 2019, 11:11 AM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Fix type annotations and squash some commits

Nov 4 2019, 11:07 AM · Origin-GNU, Lister
Harbormaster failed remote builds in B8769: Diff 7607 for D2147: lister.gnu: Standardize arguments to pass to the loader tar!
Nov 4 2019, 10:18 AM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build has FAILED

Nov 4 2019, 10:18 AM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Rebase and plug to latest master

Nov 4 2019, 10:14 AM · Origin-GNU, Lister

Oct 26 2019

lewo updated subscribers of D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

I discussed with Ludo and we agreed on the fact the current packages.json file is not really suitable for the SWH usecase.
tl;dr the idea is to expose a list of sources instead of a list of packages.

Oct 26 2019, 11:55 PM · Lister
lewo added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Since Ludo (@civodul) posted this WIP feature on guix-devel mailing list [1] I am trying to follow this thread and I would like to help.

Oct 26 2019, 10:12 PM · Lister

Oct 25 2019

zimoun added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Since Ludo (@civodul) posted this WIP feature on guix-devel mailing list [1] I am trying to follow this thread and I would like to help.

Oct 25 2019, 4:15 PM · Lister

Oct 23 2019

swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/461/ for more details.

Oct 23 2019, 1:09 PM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Rebase to latest master

Oct 23 2019, 1:06 PM · Origin-GNU, Lister

Oct 22 2019

ardumont added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Also ci job currently fails for the pep8 violations [1]:

Oct 22 2019, 11:18 AM · Lister
ardumont updated the task description for T2043: lister-phabricator: List dev.gnupg.org instance.
Oct 22 2019, 9:06 AM · Lister, Phabricator

Oct 21 2019

ardumont added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

I think my branch is almost working (i still need to rename the lister).

Oct 21 2019, 3:54 PM · Lister
Harbormaster failed remote builds in B8590: Diff 7389 for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file!
Oct 21 2019, 10:45 AM · Lister
swh-public-ci added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Build has FAILED

Oct 21 2019, 10:45 AM · Lister
lewo updated the diff for D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
  • [wip] swh.lister.json: Add lister getting sources from JSON file
  • Rebased
Oct 21 2019, 10:44 AM · Lister
lewo added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

By the way, i entertained the idea to write a guix lister with the current api, would that help (that could demo the tests to write as well)?

Oct 21 2019, 9:38 AM · Lister

Oct 20 2019

vlorentz accepted D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 20 2019, 8:56 PM · Origin-GNU, Lister

Oct 18 2019

Harbormaster failed remote builds in B8556: Diff 7357 for D2147: lister.gnu: Standardize arguments to pass to the loader tar!
Oct 18 2019, 7:45 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build was aborted

Oct 18 2019, 7:45 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build was aborted

Oct 18 2019, 7:21 PM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.
  • gnu.tree: Use .format build
Oct 18 2019, 7:04 PM · Origin-GNU, Lister
ardumont added inline comments to D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 6:45 PM · Origin-GNU, Lister
douardda accepted D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 4:59 PM · Origin-GNU, Lister
douardda added inline comments to D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 4:56 PM · Origin-GNU, Lister
Harbormaster failed remote builds in B8545: Diff 7347 for D2147: lister.gnu: Standardize arguments to pass to the loader tar!
Oct 18 2019, 3:35 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build was aborted

Oct 18 2019, 3:35 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build was aborted

Oct 18 2019, 3:07 PM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Adapt according to what's possible from the review

Oct 18 2019, 3:07 PM · Origin-GNU, Lister
ardumont added inline comments to D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 3:06 PM · Origin-GNU, Lister
ardumont added inline comments to D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 3:03 PM · Origin-GNU, Lister
vlorentz added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

I don't think this regexp is the right solution, it's quite fragile in case a package name uses one of the keywords.

Yes, it's fragile and we don't see any better solution right now.
That's how we did it initially, we improved the current version to be a tad better though.
And also it's tested even if it's limited.

A possible solution is for get_version to only accept URLs, and split the path to find the package name.

I'm not getting the difference with the current implem.
filename is extracted from the uri in any case...

Oct 18 2019, 3:01 PM · Origin-GNU, Lister
ardumont added inline comments to D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 3:00 PM · Origin-GNU, Lister
ardumont added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

I don't think this regexp is the right solution, it's quite fragile in case a package name uses one of the keywords.

Oct 18 2019, 2:54 PM · Origin-GNU, Lister
vlorentz added inline comments to D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 2:45 PM · Origin-GNU, Lister
ardumont added inline comments to D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 2:44 PM · Origin-GNU, Lister
vlorentz requested changes to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

I don't think this regexp is the right solution, it's quite fragile in case a package name uses one of the keywords. A possible solution is for get_version to only accept URLs, and split the path to find the package name.

Oct 18 2019, 2:25 PM · Origin-GNU, Lister
ardumont added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build has FAILED

Oct 18 2019, 1:19 PM · Origin-GNU, Lister
Harbormaster failed remote builds in B8540: Diff 7342 for D2147: lister.gnu: Standardize arguments to pass to the loader tar!
Oct 18 2019, 1:07 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build has FAILED

Oct 18 2019, 1:07 PM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Fix parsing timestamps

Oct 18 2019, 1:06 PM · Origin-GNU, Lister
ardumont added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build has FAILED

Oct 18 2019, 10:13 AM · Origin-GNU, Lister
ardumont updated the task description for T2032: listers: Add proper integration tests to lister.
Oct 18 2019, 9:55 AM · Lister
Harbormaster failed remote builds in B8531: Diff 7332 for D2147: lister.gnu: Standardize arguments to pass to the loader tar!
Oct 18 2019, 3:34 AM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build has FAILED

Oct 18 2019, 3:34 AM · Origin-GNU, Lister
ardumont updated the summary of D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 3:32 AM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.
  • gnu.lister: Format timestamp to isoformat string for the tar loader
  • gnu.lister: Unify timestamp formats to isoformat date in model
  • gnu.tree: Drop debug statement
  • gnu.lister: Update docstring samples
Oct 18 2019, 3:31 AM · Origin-GNU, Lister
ardumont updated the summary of D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 3:29 AM · Origin-GNU, Lister
ardumont retitled D2147: lister.gnu: Standardize arguments to pass to the loader tar from lister.gnu: Standardize arguments to pass to the loader to lister.gnu: Standardize arguments to pass to the loader tar.
Oct 18 2019, 3:24 AM · Origin-GNU, Lister
ardumont retitled D2147: lister.gnu: Standardize arguments to pass to the loader tar from lister.gnu: Move version parsing logic from loader-gnu to lister to lister.gnu: Standardize arguments to pass to the loader.
Oct 18 2019, 3:23 AM · Origin-GNU, Lister

Oct 17 2019

ardumont updated the summary of D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 17 2019, 6:14 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/446/ for more details.

Oct 17 2019, 2:20 PM · Origin-GNU, Lister
ardumont updated the diff for D2147: lister.gnu: Standardize arguments to pass to the loader tar.

Rebase to latest master

Oct 17 2019, 2:17 PM · Origin-GNU, Lister
ardumont added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

@ardumont @douardda Regarding the lister name, I agree the JSONLister name is not appropriate. @ardumont proposed FunctionalPackageManagerLister or FunctionalPackageLister but I'm not sure they are appropriated too :)

Oct 17 2019, 10:04 AM · Lister
lewo added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

@ardumont @douardda Regarding the lister name, I agree the JSONLister name is not appropriate. @ardumont proposed FunctionalPackageManagerLister or FunctionalPackageLister but I'm not sure they are appropriated too :)

Oct 17 2019, 9:41 AM · Lister
ardumont updated the summary of D2147: lister.gnu: Standardize arguments to pass to the loader tar.
Oct 17 2019, 9:41 AM · Origin-GNU, Lister

Oct 16 2019

ardumont updated the task description for T2043: lister-phabricator: List dev.gnupg.org instance.
Oct 16 2019, 10:44 AM · Lister, Phabricator
ardumont triaged T2043: lister-phabricator: List dev.gnupg.org instance as Normal priority.
Oct 16 2019, 10:43 AM · Lister, Phabricator
ardumont added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Just one note: the name is IMHO inappropriate. This is NOT a JSON lister. JSON is nothing but the serialization format used to retrieve some (more or less) well defined structured data.
What defines this lister is its ability to comprehend the very data structure mentioned in this diff's description.

Indeed, i proposed a name in D2025#inline-13301 ;)

Oct 16 2019, 10:27 AM · Lister
ardumont added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Oct 16 2019, 10:20 AM · Lister
ardumont updated the summary of D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Oct 16 2019, 10:11 AM · Lister
douardda added a comment to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.

Just one note: the name is IMHO inappropriate. This is NOT a JSON lister. JSON is nothing but the serialization format used to retrieve some (more or less) well defined structured data.

Oct 16 2019, 9:37 AM · Lister

Oct 15 2019

lewo added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Oct 15 2019, 6:26 PM · Lister
ardumont added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Oct 15 2019, 5:44 PM · Lister
ardumont added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Oct 15 2019, 5:26 PM · Lister
ardumont added inline comments to D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Oct 15 2019, 5:21 PM · Lister

Oct 13 2019

ardumont changed the status of T2032: listers: Add proper integration tests to lister from Open to Work in Progress.
Oct 13 2019, 6:37 PM · Lister

Oct 12 2019

ardumont updated the task description for T2032: listers: Add proper integration tests to lister.
Oct 12 2019, 2:59 AM · Lister

Oct 11 2019

ardumont updated the summary of D2025: [wip] swh.lister.json: Add lister getting sources from JSON file.
Oct 11 2019, 6:30 PM · Lister