Page MenuHomeSoftware Heritage
Feed Advanced Search

Oct 11 2019

ardumont updated the summary of D2025: [wip] swh.lister.functionalPackages: add lister getting sources from a JSON file.
Oct 11 2019, 6:29 PM · Lister

Oct 10 2019

ardumont updated the task description for T2032: listers: Add proper integration tests to lister.
Oct 10 2019, 9:50 AM · Lister

Oct 9 2019

ardumont closed D2074: gnu.tests: Check gnu lister.
Oct 9 2019, 5:39 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2074: gnu.tests: Check gnu lister.

Build has FAILED

Oct 9 2019, 2:48 PM · Origin-GNU, Lister
Harbormaster failed remote builds in B8219: Diff 7041 for D2074: gnu.tests: Check gnu lister!
Oct 9 2019, 2:48 PM · Origin-GNU, Lister
ardumont updated the diff for D2074: gnu.tests: Check gnu lister.

Include the tox.ini fix.

Oct 9 2019, 2:47 PM · Origin-GNU, Lister
swh-public-ci added a comment to D2074: gnu.tests: Check gnu lister.

Build has FAILED

Oct 9 2019, 2:46 PM · Origin-GNU, Lister
Harbormaster failed remote builds in B8218: Diff 7040 for D2074: gnu.tests: Check gnu lister!
Oct 9 2019, 2:46 PM · Origin-GNU, Lister
ardumont updated the diff for D2074: gnu.tests: Check gnu lister.

Update diff?

Oct 9 2019, 2:46 PM · Origin-GNU, Lister

Oct 8 2019

douardda accepted D2074: gnu.tests: Check gnu lister.
Oct 8 2019, 1:39 PM · Origin-GNU, Lister
ardumont added inline comments to D2074: gnu.tests: Check gnu lister.
Oct 8 2019, 11:08 AM · Origin-GNU, Lister
ardumont added a comment to T2032: listers: Add proper integration tests to lister.

Related D2074 D2076 D2077

Oct 8 2019, 9:06 AM · Lister
ardumont triaged T2032: listers: Add proper integration tests to lister as Normal priority.
Oct 8 2019, 9:06 AM · Lister

Oct 7 2019

ardumont added inline comments to D2074: gnu.tests: Check gnu lister.
Oct 7 2019, 2:28 PM · Origin-GNU, Lister
ardumont added inline comments to D2074: gnu.tests: Check gnu lister.
Oct 7 2019, 10:33 AM · Origin-GNU, Lister
douardda requested changes to D2074: gnu.tests: Check gnu lister.
Oct 7 2019, 10:24 AM · Origin-GNU, Lister

Oct 5 2019

ardumont updated the summary of D2074: gnu.tests: Check gnu lister.
Oct 5 2019, 2:25 PM · Origin-GNU, Lister
ardumont retitled D2074: gnu.tests: Check gnu lister from gnu.tests: Check gnu listing to gnu.tests: Check gnu lister.
Oct 5 2019, 12:56 PM · Origin-GNU, Lister
ardumont retitled D2074: gnu.tests: Check gnu lister from gnu.tests: Add basic listing check from a tree.json.gz to gnu.tests: Check gnu listing.
Oct 5 2019, 12:56 PM · Origin-GNU, Lister

Sep 20 2019

ardumont closed D2014: swh-lister: Remove deprecated console scripts `swh-lister`.
Sep 20 2019, 11:21 AM · SWH command line interface, Lister
anlambert accepted D2014: swh-lister: Remove deprecated console scripts `swh-lister`.
Sep 20 2019, 11:18 AM · SWH command line interface, Lister
swh-public-ci added a comment to D2014: swh-lister: Remove deprecated console scripts `swh-lister`.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/375/ for more details.

Sep 20 2019, 11:15 AM · SWH command line interface, Lister
ardumont retitled D2014: swh-lister: Remove deprecated console scripts `swh-lister` from setup.py: Fix wrong console scripts entry point for swh-lister command to swh-lister: Remove deprecated console scripts `swh-lister`.
Sep 20 2019, 11:13 AM · SWH command line interface, Lister
ardumont updated the diff for D2014: swh-lister: Remove deprecated console scripts `swh-lister`.

In the end, kill the deprecated and unused command

Sep 20 2019, 11:12 AM · SWH command line interface, Lister
swh-public-ci added a comment to D2014: swh-lister: Remove deprecated console scripts `swh-lister`.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/374/ for more details.

Sep 20 2019, 10:05 AM · SWH command line interface, Lister
ardumont updated the test plan for D2014: swh-lister: Remove deprecated console scripts `swh-lister`.
Sep 20 2019, 10:04 AM · SWH command line interface, Lister

Sep 17 2019

anlambert closed T2000: Phabricator lister erroneously delete some loading tasks as Resolved by committing rDLS4c8d7baf7591: phabricator/lister: Prevent erroneous scheduler tasks disabling.
Sep 17 2019, 10:59 AM · Phabricator forge, Lister
anlambert closed T1997: lister-phabricator: Pagination error as Resolved by committing rDLSe83902c2a36a: phabricator/lister: Fix get_next_target_from_response return type.
Sep 17 2019, 10:52 AM · Phabricator, Lister
anlambert closed T1999: Phabricator lister only creates one loading task as Resolved by committing rDLS1ebe762ea6f3: phabricator/lister: Do not override max_index when bootstrapping.
Sep 17 2019, 10:51 AM · Phabricator forge, Lister

Sep 16 2019

anlambert added a revision to T2000: Phabricator lister erroneously delete some loading tasks : D1990: Phabricator: Prevent erroneous scheduler tasks disabling.
Sep 16 2019, 8:27 PM · Phabricator forge, Lister
anlambert added a revision to T1997: lister-phabricator: Pagination error: D1984: Phabricator: Fix get_next_target_from_response return type.
Sep 16 2019, 1:38 PM · Phabricator, Lister
anlambert added a project to T1999: Phabricator lister only creates one loading task : Phabricator forge.
Sep 16 2019, 12:20 PM · Phabricator forge, Lister
anlambert triaged T2000: Phabricator lister erroneously delete some loading tasks as Normal priority.
Sep 16 2019, 12:20 PM · Phabricator forge, Lister
anlambert created T2000: Phabricator lister erroneously delete some loading tasks .
Sep 16 2019, 12:20 PM · Phabricator forge, Lister
anlambert triaged T1999: Phabricator lister only creates one loading task as High priority.
Sep 16 2019, 12:01 PM · Phabricator forge, Lister

Sep 12 2019

ardumont triaged T1997: lister-phabricator: Pagination error as Normal priority.
Sep 12 2019, 4:01 PM · Phabricator, Lister
ardumont closed T1996: swh-lister: Phabricator lister no longer works as Resolved.
Sep 12 2019, 3:56 PM · Phabricator, Lister
ardumont added a comment to T1996: swh-lister: Phabricator lister no longer works.

Related https://forge.softwareheritage.org/transactions/detail/PHID-XACT-PSTE-j46unxzdjtok26u/

Sep 12 2019, 3:54 PM · Phabricator, Lister
ardumont edited P529 migrate swh-lister tasks (in scheduler db) following the lister normalization.
Sep 12 2019, 3:54 PM · Lister
ardumont added a comment to T1996: swh-lister: Phabricator lister no longer works.

So like i replied to an email early on (which did not caught up here), the problem lies with the input.
So no real bug whatsoever!

Sep 12 2019, 3:44 PM · Phabricator, Lister
ardumont renamed T1996: swh-lister: Phabricator lister no longer works from staging/docker-dev: Phabricator lister no longer works to swh-lister: Phabricator lister no longer works.
Sep 12 2019, 10:15 AM · Phabricator, Lister

Sep 11 2019

ardumont closed T1861: cgit lister: Adapt lister to deal with inconsistent `git clone uri` pattern as Resolved.

D1929 took care of it ;)

Sep 11 2019, 4:12 PM · CGit lister, Lister
ardumont added a comment to P529 migrate swh-lister tasks (in scheduler db) following the lister normalization.

Related D1950
Related D1929
Related T1861

Sep 11 2019, 4:03 PM · Lister
ardumont added a project to P529 migrate swh-lister tasks (in scheduler db) following the lister normalization: Lister.
Sep 11 2019, 4:02 PM · Lister
ardumont added a comment to T1996: swh-lister: Phabricator lister no longer works.

As for the title, it no longer works as we deployed listing phabricator forges in the past, see for example kde's [1] or our own [2]

Sep 11 2019, 3:08 PM · Phabricator, Lister
ardumont added a comment to T1996: swh-lister: Phabricator lister no longer works.

(This may be irrelevant to our latest refactorings though)

Sep 11 2019, 3:06 PM · Phabricator, Lister
ardumont updated the task description for T1996: swh-lister: Phabricator lister no longer works.
Sep 11 2019, 2:53 PM · Phabricator, Lister
ardumont triaged T1996: swh-lister: Phabricator lister no longer works as Normal priority.
Sep 11 2019, 2:53 PM · Phabricator, Lister
ardumont added a comment to T1995: swh-lister: Unstuck debian build failures.

Another fix to actually reference the python3-swh.storage.schemata runtime dependency.
Without this, a fresh lister install makes the service fail to start as well.

Sep 11 2019, 11:33 AM · Lister, System administration

Sep 10 2019

olasd added a comment to T1995: swh-lister: Unstuck debian build failures.

With the proper version constraint (dh-python (>= 3)) the package build went through for all three suites (unstable, stretch, buster).

Sep 10 2019, 3:11 PM · Lister, System administration
ardumont added a comment to T1995: swh-lister: Unstuck debian build failures.

For others (including future me) [1]

Sep 10 2019, 3:06 PM · Lister, System administration
ardumont added a comment to T1995: swh-lister: Unstuck debian build failures.

Thanks.

Sep 10 2019, 2:57 PM · Lister, System administration
olasd added a comment to T1995: swh-lister: Unstuck debian build failures.

By default, pybuild doesn't copy the egg-info files (which contains the entry points) before running tests. This means the entrypoints aren't available.

Sep 10 2019, 2:54 PM · Lister, System administration
olasd closed T1995: swh-lister: Unstuck debian build failures as Resolved by committing rDLSc728abe7ad70: Add egg-info to pybuild.testfiles.
Sep 10 2019, 2:51 PM · Lister, System administration
ardumont added a comment to T1995: swh-lister: Unstuck debian build failures.

Editing our swh.scheduler.tests.conftest's @celery_includes definition fixture (to print the task_modules computed).
We can further see that the task modules computation is off:

Sep 10 2019, 2:21 PM · Lister, System administration
ardumont added a project to T1995: swh-lister: Unstuck debian build failures: Lister.
Sep 10 2019, 2:00 PM · Lister, System administration

Sep 9 2019

ardumont triaged T1991: Implement a Guix/Nix loader as Normal priority.
Sep 9 2019, 10:08 AM · Package Loader

Sep 6 2019

ardumont added a comment to T1859: bitbucket lister: Deal with connection errors.

They are still working on it...
(last update from 04/09/2019 mentioning it's still wip on their dev team side).

Sep 6 2019, 10:02 AM · Lister, Bitbucket lister

Sep 2 2019

ardumont added a comment to T1979: Remarks on the tutorial "Run a new lister" .

Lister's input are forge's url (among other things).
Lister's output are scheduling tasks (for loader) in the scheduler db (with their arguments set for the loader to take it on).

Sep 2 2019, 7:44 AM · Documentation, Lister

Sep 1 2019

nahimilega updated the task description for T1946: Improve run_a_new_lister.rst file.
Sep 1 2019, 10:07 PM · Easy hack, Documentation, Lister
lewo added a comment to T1979: Remarks on the tutorial "Run a new lister" .

Ok, thx a lot for your answers!

Sep 1 2019, 9:22 PM · Documentation, Lister
ardumont added a comment to T1979: Remarks on the tutorial "Run a new lister" .

Thanks, this is highly appreciated feedback ;)

Sep 1 2019, 11:05 AM · Documentation, Lister
zack triaged T1979: Remarks on the tutorial "Run a new lister" as Normal priority.
Sep 1 2019, 10:45 AM · Documentation, Lister

Aug 30 2019

douardda triaged T1978: Simplify listers's db model as Low priority.
Aug 30 2019, 6:12 PM · Lister

Aug 29 2019

ardumont updated the summary of D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 29 2019, 11:00 AM · Lister
ardumont updated the summary of D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 29 2019, 10:59 AM · Lister

Aug 28 2019

ardumont closed D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 4:33 PM · Lister
swh-public-ci added a comment to D1919: listers: Open cli to allow a first listing pass on new forge instance.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/307/ for more details.

Aug 28 2019, 4:32 PM · Lister
ardumont updated the diff for D1919: listers: Open cli to allow a first listing pass on new forge instance.

Rework commits (squash some)

Aug 28 2019, 4:30 PM · Lister
douardda accepted D1919: listers: Open cli to allow a first listing pass on new forge instance.

LGTM

Aug 28 2019, 3:57 PM · Lister
ardumont added inline comments to D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 3:24 PM · Lister
ardumont updated the summary of D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 3:24 PM · Lister
ardumont updated the summary of D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 3:23 PM · Lister
swh-public-ci added a comment to D1919: listers: Open cli to allow a first listing pass on new forge instance.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/306/ for more details.

Aug 28 2019, 3:22 PM · Lister
ardumont updated the summary of D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 3:21 PM · Lister
ardumont updated the diff for D1919: listers: Open cli to allow a first listing pass on new forge instance.

Adapt according to reviews/discussion:

Aug 28 2019, 3:20 PM · Lister
douardda added inline comments to D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 3:06 PM · Lister
ardumont added a comment to D1919: listers: Open cli to allow a first listing pass on new forge instance.

Thanks for your feedback all ;)

Aug 28 2019, 3:03 PM · Lister
douardda added inline comments to D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 2:59 PM · Lister
swh-public-ci added a comment to D1919: listers: Open cli to allow a first listing pass on new forge instance.

Build is green
See https://jenkins.softwareheritage.org/job/DLS/job/tox/305/ for more details.

Aug 28 2019, 2:58 PM · Lister
douardda added inline comments to D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 2:57 PM · Lister
ardumont updated the diff for D1919: listers: Open cli to allow a first listing pass on new forge instance.
  • cli: Bootstrap tests on cli
Aug 28 2019, 2:56 PM · Lister
vlorentz added a comment to D1919: listers: Open cli to allow a first listing pass on new forge instance.

Looking into how to add tests to the cli.

Aug 28 2019, 2:09 PM · Lister
ardumont added a comment to D1919: listers: Open cli to allow a first listing pass on new forge instance.

Looking into how to add tests to the cli.
That should not block a first review pass though ;)

Aug 28 2019, 2:02 PM · Lister
ardumont updated the summary of D1919: listers: Open cli to allow a first listing pass on new forge instance.
Aug 28 2019, 1:48 PM · Lister
ardumont added a project to D1919: listers: Open cli to allow a first listing pass on new forge instance: Lister.
Aug 28 2019, 12:16 PM · Lister
ardumont added a comment to T1859: bitbucket lister: Deal with connection errors.

(they apparently allows the user to re-open but when actually trying to, it's not working...).

Aug 28 2019, 10:06 AM · Lister, Bitbucket lister

Aug 26 2019

ardumont added a comment to T1859: bitbucket lister: Deal with connection errors.

Ticket [1] is closed because i did not reply fast enough.

Aug 26 2019, 12:25 PM · Lister, Bitbucket lister
ardumont updated the title for P459 ugly bitbucket lister workaround from ugly unstucking bitbucket lister to ugly bitbucket lister workaround.
Aug 26 2019, 10:33 AM · Origin-Bitbucket, Bitbucket lister, Lister
ardumont updated the task description for T1859: bitbucket lister: Deal with connection errors.
Aug 26 2019, 10:31 AM · Lister, Bitbucket lister

Aug 22 2019

vlorentz added a project to T1946: Improve run_a_new_lister.rst file: Easy hack.
Aug 22 2019, 12:30 PM · Easy hack, Documentation, Lister
vlorentz added a project to T1946: Improve run_a_new_lister.rst file: Documentation.
Aug 22 2019, 12:29 PM · Easy hack, Documentation, Lister

Aug 12 2019

vlorentz added a comment to T1947: Unsupported locale 'en_IN' in arrow lib.

Although I feel this is more of arrow library issue rather than swh issue.

Aug 12 2019, 4:08 PM · Scheduling utilities
GoodwillHunter added a comment to T1947: Unsupported locale 'en_IN' in arrow lib.

Thanks @nahimilega, changing locale worked out fine

Aug 12 2019, 3:30 PM · Scheduling utilities

Aug 8 2019

nahimilega updated subscribers of T1947: Unsupported locale 'en_IN' in arrow lib.

I solved this issue by changing my system locale to 'en_US'.
You can refer to these tutorials to change your system
https://www.tecmint.com/set-system-locales-in-linux/
https://askubuntu.com/questions/89976/how-do-i-change-the-default-locale-in-ubuntu-server

Aug 8 2019, 10:28 PM · Scheduling utilities
GoodwillHunter triaged T1947: Unsupported locale 'en_IN' in arrow lib as Normal priority.
Aug 8 2019, 10:10 PM · Scheduling utilities
nahimilega added a project to T1946: Improve run_a_new_lister.rst file: Lister.
Aug 8 2019, 10:00 PM · Easy hack, Documentation, Lister

Jul 19 2019

ardumont closed T1923: Implement Packagist Lister, a subtask of T1776: packagist (PHP) Lister, as Resolved.
Jul 19 2019, 4:48 PM · Lister, Archive coverage
ardumont closed T1923: Implement Packagist Lister as Resolved.
Jul 19 2019, 4:48 PM · Lister, Archive coverage
ardumont added a revision to T1923: Implement Packagist Lister: D1584: swh.lister.packagist.
Jul 19 2019, 4:36 PM · Lister, Archive coverage