Page MenuHomeSoftware Heritage
Feed Advanced Search

Oct 27 2022

anlambert closed T4665: [pudbev lister] use alternative packages list endpoint, a subtask of T4465: Ingest pub.dev (Dart, Flutter), as Invalid.
Oct 27 2022, 4:41 PM · PubDev loader, PubDev lister, Archive coverage
bchauvet added a subtask for T4465: Ingest pub.dev (Dart, Flutter): T4665: [pudbev lister] use alternative packages list endpoint.
Oct 27 2022, 2:14 PM · PubDev loader, PubDev lister, Archive coverage

Oct 21 2022

anlambert edited projects for T4448: Implementation of Fedora Lister, added: RPM lister; removed Lister.
Oct 21 2022, 11:38 AM · RPM lister, Archive coverage
bchauvet edited parent tasks for T4448: Implementation of Fedora Lister, added: Unknown Object (Maniphest Task); removed: Unknown Object (Maniphest Task).
Oct 21 2022, 9:55 AM · RPM lister, Archive coverage

Oct 19 2022

gitlab-migration closed T4626: staging: ingest openbsd.org cvs forge, a subtask of T3691: Implement CVS loader, as Migrated.
Oct 19 2022, 6:08 PM · CVS loader, Archive coverage
gitlab-migration closed T4626: staging: ingest openbsd.org cvs forge as Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:08 PM · System administration, Archive coverage
gitlab-migration closed T4625: staging: ingest netbsd.org cvs forge as Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:08 PM · System administration, Archive coverage
gitlab-migration closed T4625: staging: ingest netbsd.org cvs forge, a subtask of T3691: Implement CVS loader, as Migrated.
Oct 19 2022, 6:08 PM · CVS loader, Archive coverage
gitlab-migration changed the status of T4555: staging: Deploy bower lister from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:08 PM · System administration, Bower lister, Archive coverage
gitlab-migration changed the status of T4555: staging: Deploy bower lister, a subtask of T4475: Ingest bower.io (Javascript package manager), from Resolved to Migrated.
Oct 19 2022, 6:08 PM · Bower lister, Archive coverage
gitlab-migration changed the status of T4545: staging: Deploy golang stack, a subtask of T4124: Golang support, from Resolved to Migrated.
Oct 19 2022, 6:08 PM · Golang loader, Golang lister, Archive coverage
gitlab-migration changed the status of T4545: staging: Deploy golang stack from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:08 PM · System administration, Golang loader, Golang lister, Archive coverage
gitlab-migration changed the status of T4517: staging: Deploy pubdev, a subtask of T4465: Ingest pub.dev (Dart, Flutter), from Resolved to Migrated.
Oct 19 2022, 6:08 PM · PubDev loader, PubDev lister, Archive coverage
gitlab-migration changed the status of T4517: staging: Deploy pubdev from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:08 PM · System administration, PubDev loader, PubDev lister, Archive coverage
gitlab-migration changed the status of T4478: Deploy Gogs lister to staging, a subtask of T1721: Implementation of Gogs Lister, from Resolved to Migrated.
Oct 19 2022, 6:08 PM · Gogs lister, Origin-Gitea/Gogs, Archive coverage, Lister
gitlab-migration changed the status of T4330: Deploy maven stack in production from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:07 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
gitlab-migration changed the status of T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, from Resolved to Migrated.
Oct 19 2022, 6:07 PM · Maven loader, Maven lister, GSoC 2019, Archive coverage
gitlab-migration changed the status of T4326: Archive the pom file additionally to the source folder, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Invalid to Migrated.
Oct 19 2022, 6:07 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4326: Archive the pom file additionally to the source folder from Invalid to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:07 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4215: staging: Deploy latest maven stack, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
Oct 19 2022, 6:06 PM · Maven loader, Maven lister, Archive coverage
gitlab-migration changed the status of T4215: staging: Deploy latest maven stack from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4143: staging: Deploy maven stack fixes, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
Oct 19 2022, 6:06 PM · Maven loader, Maven lister, Archive coverage
gitlab-migration changed the status of T4143: staging: Deploy maven stack fixes from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4105: Push maven-index-exporter image to docker hub, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Resolved to Migrated.
Oct 19 2022, 6:06 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T3947: Deploy swh.lister v2.7, a subtask of T3789: Adapt sourceforge lister to list cvs origins according to what the cvs loader expects, from Resolved to Migrated.
Oct 19 2022, 6:06 PM · CVS loader, Archive coverage
gitlab-migration changed the status of T3798: Debian package for swh.loader.cvs, a subtask of T3788: staging: Deploy cvs loader v0.1, from Resolved to Migrated.
Oct 19 2022, 6:05 PM · System administration, CVS loader, Archive coverage
gitlab-migration changed the status of T3798: Debian package for swh.loader.cvs from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:05 PM · System administration, CVS loader, Archive coverage
gitlab-migration changed the status of T3788: staging: Deploy cvs loader v0.1, a subtask of T3691: Implement CVS loader, from Resolved to Migrated.
Oct 19 2022, 6:05 PM · CVS loader, Archive coverage
gitlab-migration changed the status of T3788: staging: Deploy cvs loader v0.1 from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:05 PM · System administration, CVS loader, Archive coverage
gitlab-migration changed the status of T3785: Update scheduler metrics routine is taking too long from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:05 PM · Archive coverage, System administration
gitlab-migration changed the status of T3774: Deploy swh.loader.core v2.0.0, a subtask of T2400: Ingest current and historical Ubuntu releases, from Resolved to Migrated.
Oct 19 2022, 6:05 PM · System administration, Debian loader, Package Loader, Archive coverage
gitlab-migration changed the status of T3765: Deploy latest swh.loader.core and swh.lister, a subtask of T2400: Ingest current and historical Ubuntu releases, from Resolved to Migrated.
Oct 19 2022, 6:05 PM · System administration, Debian loader, Package Loader, Archive coverage
gitlab-migration changed the status of T3746: staging: Deploy maven indexer/lister/loader, a subtask of T1724: Maven Central repository support, from Resolved to Migrated.
Oct 19 2022, 6:05 PM · Maven loader, Maven lister, GSoC 2019, Archive coverage
gitlab-migration changed the status of T3746: staging: Deploy maven indexer/lister/loader from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:05 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T3717: Ingest opam instance https://coq.inria.fr/opam/released/ from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:04 PM · System administration, Archive coverage, Opam
gitlab-migration changed the status of T3667: Orchestrate origins scheduling according to scheduler metrics feedback from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:04 PM · System administration, Sprint 2021 01, Archive coverage, Scheduling utilities
gitlab-migration changed the status of T3667: Orchestrate origins scheduling according to scheduler metrics feedback, a subtask of T2345: Improve handling of recurrent loading tasks in scheduler, from Resolved to Migrated.
Oct 19 2022, 6:04 PM · Sprint 2021 01, Archive coverage, Scheduling utilities
gitlab-migration changed the status of T3600: List heptapod instance https://heptapod.host/, a subtask of T3599: List and ingest heptapod instances, from Resolved to Migrated.
Oct 19 2022, 6:04 PM · System administration, Archive coverage
gitlab-migration changed the status of T3600: List heptapod instance https://heptapod.host/ from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:04 PM · System administration, Archive coverage
gitlab-migration changed the status of T3599: List and ingest heptapod instances from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:04 PM · System administration, Archive coverage
gitlab-migration changed the status of T3597: List heptapod instance https://forge.extranet.logilab.fr/, a subtask of T3599: List and ingest heptapod instances, from Resolved to Migrated.
Oct 19 2022, 6:04 PM · System administration, Archive coverage
gitlab-migration changed the status of T3597: List heptapod instance https://forge.extranet.logilab.fr/ from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:04 PM · System administration, Archive coverage
gitlab-migration changed the status of T3581: List heptapod instance foss.heptapod.net, a subtask of T3599: List and ingest heptapod instances, from Resolved to Migrated.
Oct 19 2022, 6:04 PM · System administration, Archive coverage
gitlab-migration changed the status of T3581: List heptapod instance foss.heptapod.net from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:04 PM · Archive coverage, System administration, Origin-GitLab
gitlab-migration changed the status of T3568: Deploy opam lister/loader to production, a subtask of T3424: Opam support, from Resolved to Migrated.
Oct 19 2022, 6:04 PM · Archive coverage, Opam
gitlab-migration changed the status of T3568: Deploy opam lister/loader to production from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:04 PM · System administration, Archive coverage, Opam
gitlab-migration changed the status of T3538: Send scheduler metrics to prometheus, a subtask of T2345: Improve handling of recurrent loading tasks in scheduler, from Resolved to Migrated.
Oct 19 2022, 6:04 PM · Sprint 2021 01, Archive coverage, Scheduling utilities
gitlab-migration changed the status of T3471: production: Deploy swh.scheduler v0.17, a subtask of T2345: Improve handling of recurrent loading tasks in scheduler, from Resolved to Migrated.
Oct 19 2022, 6:03 PM · Sprint 2021 01, Archive coverage, Scheduling utilities
gitlab-migration changed the status of T3471: production: Deploy swh.scheduler v0.17 from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:03 PM · System administration, Archive coverage, Scheduling utilities
gitlab-migration changed the status of T3470: lister-sourceforge: Activate sourceforge origins when listed, a subtask of T3374: Ingest sourceforge repositories (origins of type git, svn, hg), from Resolved to Migrated.
Oct 19 2022, 6:03 PM · System administration, Archive coverage, Origin-SourceForge
gitlab-migration changed the status of T3470: lister-sourceforge: Activate sourceforge origins when listed from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:03 PM · System administration, Archive coverage, Origin-SourceForge
gitlab-migration changed the status of T3468: staging: current opam loading issues, a subtask of T3424: Opam support, from Resolved to Migrated.
Oct 19 2022, 6:03 PM · Archive coverage, Opam
gitlab-migration changed the status of T3456: staging: Deploy scheduler v0.17 from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:03 PM · System administration, Sprint 2021 01, Archive coverage, Scheduling utilities
gitlab-migration changed the status of T3456: staging: Deploy scheduler v0.17, a subtask of T2345: Improve handling of recurrent loading tasks in scheduler, from Resolved to Migrated.
Oct 19 2022, 6:03 PM · Sprint 2021 01, Archive coverage, Scheduling utilities
gitlab-migration changed the status of T3438: Deploy opam lister/loader to staging, a subtask of T3424: Opam support, from Resolved to Migrated.
Oct 19 2022, 6:03 PM · Archive coverage, Opam
gitlab-migration changed the status of T3374: Ingest sourceforge repositories (origins of type git, svn, hg) from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:02 PM · System administration, Archive coverage, Origin-SourceForge
gitlab-migration changed the status of T3350: Deploy sourceforge lister in production from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:02 PM · Archive coverage, System administration, Origin-SourceForge
gitlab-migration changed the status of T2608: Deploy launchpad and gitea listers on production, a subtask of T1734: Create a Lister for launchpad.net, from Resolved to Migrated.
Oct 19 2022, 5:58 PM · Lister, Archive coverage
gitlab-migration changed the status of T2608: Deploy launchpad and gitea listers on production, a subtask of T2313: Archive git.fsfe.org (Gitea), from Resolved to Migrated.
Oct 19 2022, 5:58 PM · Archive coverage, Lister
gitlab-migration changed the status of T2608: Deploy launchpad and gitea listers on production from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:58 PM · Archive coverage, System administration
gitlab-migration changed the status of T2594: production: Running nixguix on guix sources, a subtask of T1352: ingest Guix (SD) packages, from Resolved to Migrated.
Oct 19 2022, 5:58 PM · Archive coverage
gitlab-migration changed the status of T2594: production: Running nixguix on guix sources from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:58 PM · Archive coverage, System administration
gitlab-migration changed the status of T2594: production: Running nixguix on guix sources, a subtask of T2485: staging: Running nixguix on guix sources , from Resolved to Migrated.
Oct 19 2022, 5:58 PM · Archive coverage
gitlab-migration changed the status of T2400: Ingest current and historical Ubuntu releases from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:57 PM · System administration, Debian loader, Package Loader, Archive coverage
gitlab-migration changed the status of T2358: Deploy launchpad lister on staging from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:57 PM · System administration, Lister, Archive coverage
gitlab-migration changed the status of T2358: Deploy launchpad lister on staging, a subtask of T1734: Create a Lister for launchpad.net, from Resolved to Migrated.
Oct 19 2022, 5:57 PM · Lister, Archive coverage
gitlab-migration changed the status of T360: create gid 5000 and add swhworker to it (to ingest gitorious repos), a subtask of T312: Gitorious import: ingest repositories, from Resolved to Migrated.
Oct 19 2022, 5:51 PM · Archive coverage, Restricted Project, Origin-Gitorious, Format-Git

Oct 18 2022

anlambert added a comment to T4625: staging: ingest netbsd.org cvs forge.

@vsellier, I landed all optimizations for the CVS loader and tagged a new version v0.5.0 so you can retry the NetBSD repository loading on staging.

Oct 18 2022, 1:32 PM · System administration, Archive coverage

Oct 17 2022

anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8689: docs: Add info about CPAN extrinsic metadata format.
Oct 17 2022, 5:43 PM · CPAN lister, Archive coverage
anlambert closed Restricted Maniphest Task, a subtask of T4625: staging: ingest netbsd.org cvs forge, as Resolved.
Oct 17 2022, 10:55 AM · System administration, Archive coverage

Oct 14 2022

anlambert added a comment to T4625: staging: ingest netbsd.org cvs forge.

@vsellier, I found other memory consumption issues in CVS loader implementation while testing the parsing of the huge rlog output of NetBSD (script got OOM killed). I fixed those in D8683.

Oct 14 2022, 11:44 AM · System administration, Archive coverage
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

thanks, I will try to restart the loading with the patch to see how it behaves

Oct 14 2022, 8:57 AM · System administration, Archive coverage

Oct 13 2022

anlambert added a comment to T4625: staging: ingest netbsd.org cvs forge.

The loader got killed after it starts to consume a lot of memory...

Warning: Permanently added 'anoncvs.netbsd.org,199.233.217.198' (RSA) to the list of known hosts.
DEBUG:swh.loader.cvs.loader.CvsLoader:Fetching CVS rlog from anoncvs.netbsd.org:/cvsroot/src
Killed
swh@loader-cvs-manual:~$
Oct 13 2022, 7:36 PM · System administration, Archive coverage
anlambert closed Restricted Maniphest Task, a subtask of T4625: staging: ingest netbsd.org cvs forge, as Resolved.
Oct 13 2022, 7:31 PM · System administration, Archive coverage
vlorentz triaged T4632: Origin URLs generated for Fedora origins as Normal priority.
Oct 13 2022, 10:40 AM · Archive coverage, Data Model
vlorentz added subtasks for T4625: staging: ingest netbsd.org cvs forge: Restricted Maniphest Task, Restricted Maniphest Task.
Oct 13 2022, 10:23 AM · System administration, Archive coverage
vlorentz updated the task description for T4628: Pubdev: load authors from API instead of pubspec.yaml.
Oct 13 2022, 10:15 AM · PubDev loader, PubDev lister, Archive coverage
vlorentz triaged T4628: Pubdev: load authors from API instead of pubspec.yaml as Normal priority.
Oct 13 2022, 10:15 AM · PubDev loader, PubDev lister, Archive coverage
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

The loading finally failed:

INFO:swh.loader.cvs.loader.CvsLoader:Load origin 'ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src' with type 'cvs'
DEBUG:swh.loader.cvs.loader.CvsLoader:lister_not provided, skipping extrinsic origin metadata
DEBUG:swh.loader.cvs.loader.CvsLoader:prepare; origin_url=ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src scheme=ssh path=/cvsroot/src
Warning: Permanently added 'anoncvs.netbsd.org,199.233.217.198' (RSA) to the list of known hosts.
DEBUG:swh.loader.cvs.loader.CvsLoader:Fetching CVS rlog from anoncvs.netbsd.org:/cvsroot/src
Oct 13 2022, 9:21 AM · System administration, Archive coverage

Oct 12 2022

vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

I've launched the loader on the /src subdirectory, if I follow the cvs loader documentation: https://forge.softwareheritage.org/source/swh-loader-cvs/browse/master/docs/ ,
it should also be launched on the other subdirectories:

Oct 12 2022, 6:44 PM · System administration, Archive coverage
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

The loader got killed after it starts to consume a lot of memory...

Warning: Permanently added 'anoncvs.netbsd.org,199.233.217.198' (RSA) to the list of known hosts.
DEBUG:swh.loader.cvs.loader.CvsLoader:Fetching CVS rlog from anoncvs.netbsd.org:/cvsroot/src
Killed
swh@loader-cvs-manual:~$
Oct 12 2022, 6:20 PM · System administration, Archive coverage
vlorentz updated the task description for T4623: NuGet: Store source links.
Oct 12 2022, 2:03 PM · Archive coverage
vlorentz added a revision to T4465: Ingest pub.dev (Dart, Flutter): D8665: Pubdev: Add raw_extrinsic_metadata.
Oct 12 2022, 1:26 PM · PubDev loader, PubDev lister, Archive coverage
vsellier moved T4626: staging: ingest openbsd.org cvs forge from Backlog to in-progress on the System administration board.
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier moved T4625: staging: ingest netbsd.org cvs forge from Backlog to in-progress on the System administration board.
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier added a project to T4625: staging: ingest netbsd.org cvs forge: System administration.
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier added a comment to T4626: staging: ingest openbsd.org cvs forge.

Manual loading in progress:

  • pod created with P1494
  • manual loading:
Oct 12 2022, 11:45 AM · System administration, Archive coverage
vsellier added a comment to T4625: staging: ingest netbsd.org cvs forge.

The first try with a oneshot task failed, the containe was killed:

swhscheduler@scheduler0:~$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ task add load-cvs -p oneshot ssh://anoncvs@anoncvs.NetBSD.org:/cvsroot/src
Created 1 tasks
Oct 12 2022, 11:43 AM · System administration, Archive coverage
vsellier changed the status of T4626: staging: ingest openbsd.org cvs forge from Open to Work in Progress.
Oct 12 2022, 11:36 AM · System administration, Archive coverage
vsellier changed the status of T4625: staging: ingest netbsd.org cvs forge from Open to Work in Progress.
Oct 12 2022, 11:35 AM · System administration, Archive coverage
vlorentz triaged T4624: NuGet: Store source packages as Normal priority.
Oct 12 2022, 10:33 AM · Archive coverage
vlorentz triaged T4623: NuGet: Store source links as Normal priority.
Oct 12 2022, 10:31 AM · Archive coverage

Oct 11 2022

anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8661: tarball: fallback using tar command when shutil.unpack_archive failed.
Oct 11 2022, 6:05 PM · CPAN lister, Archive coverage

Oct 10 2022

anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8652: cpan: Collect extrinsic metadata for each module release.
Oct 10 2022, 5:04 PM · CPAN lister, Archive coverage
anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8651: cpan: Do not parse intrinsic metadata for getting module author.
Oct 10 2022, 5:04 PM · CPAN lister, Archive coverage
anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8649: cpan: Use a fake origin URL instead of an HTTP one.
Oct 10 2022, 4:32 PM · CPAN lister, Archive coverage
anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8648: cpan: Fix module version extraction for some edge cases.
Oct 10 2022, 4:26 PM · CPAN lister, Archive coverage

Oct 7 2022

anlambert added a revision to T1777: Rubygems Lister: D8639: rubygems: Use gems database dump to improve listing output.
Oct 7 2022, 11:54 AM · RubyGems lister, Archive coverage

Oct 4 2022

anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8616: cpan: Align loader implementation with latest lister improvements.
Oct 4 2022, 5:24 PM · CPAN lister, Archive coverage
anlambert added a revision to T2833: cpan.loader - archive Perl modules from CPAN: D8615: cpan: Improve listing process by querying the metacpan release endpoint.
Oct 4 2022, 5:17 PM · CPAN lister, Archive coverage