Page MenuHomeSoftware Heritage
Feed Advanced Search

May 18 2020

ardumont removed a revision from T1398: npm incremental lister: D3162: admin/deposit: Extract origin from swh_anchor_id according to latest change.
May 18 2020, 2:07 PM · Npm Lister, Origin-npm
ardumont added a revision to T1398: npm incremental lister: D3162: admin/deposit: Extract origin from swh_anchor_id according to latest change.
May 18 2020, 1:57 PM · Npm Lister, Origin-npm

May 9 2020

rdicosmo moved T1766: Write blog post about the npm ingestion from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.
May 9 2020, 5:33 PM · Origin-npm, Blog, Website, Unknown Object (Project)

Nov 26 2019

ardumont closed T2098: Deploy package loaders as Resolved.
Nov 26 2019, 5:28 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont updated the task description for T2098: Deploy package loaders.
Nov 26 2019, 5:28 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont closed T1389: Implement a base "package" loader for package managers as Resolved.
Nov 26 2019, 12:25 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont removed a parent task for T2098: Deploy package loaders: T1389: Implement a base "package" loader for package managers.
Nov 26 2019, 12:24 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont removed a subtask for T1389: Implement a base "package" loader for package managers: T2098: Deploy package loaders.
Nov 26 2019, 12:24 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont updated the task description for T2098: Deploy package loaders.
Nov 26 2019, 12:13 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Nov 22 2019

ardumont closed D2344: npm: Explicitely retrieve the revision date from extrinsic metadata.
Nov 22 2019, 6:11 PM · Origin-npm, Npm loader, Core Loader
olasd accepted D2344: npm: Explicitely retrieve the revision date from extrinsic metadata.
Nov 22 2019, 6:10 PM · Origin-npm, Npm loader, Core Loader
ardumont updated the test plan for D2344: npm: Explicitely retrieve the revision date from extrinsic metadata.
Nov 22 2019, 6:08 PM · Origin-npm, Npm loader, Core Loader

Nov 20 2019

ardumont updated the task description for T2098: Deploy package loaders.
Nov 20 2019, 9:42 AM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Nov 19 2019

ardumont removed a subtask for T1389: Implement a base "package" loader for package managers: T2026: Implement cran loader with package manager mechanism.
Nov 19 2019, 12:27 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont closed T2025: Re-implement debian loader with package loader mechanism, a subtask of T1389: Implement a base "package" loader for package managers, as Resolved.
Nov 19 2019, 12:26 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont closed T2024: Re-implement deposit loader with package loader mechanism, a subtask of T1389: Implement a base "package" loader for package managers, as Resolved.
Nov 19 2019, 12:26 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont closed T2023: Re-implement gnu loader with package loader, a subtask of T1389: Implement a base "package" loader for package managers, as Resolved.
Nov 19 2019, 12:26 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont closed T2022: Re-implement npm loader with base loader, a subtask of T1389: Implement a base "package" loader for package managers, as Resolved.
Nov 19 2019, 12:25 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont closed T2022: Re-implement npm loader with base loader as Resolved.
Nov 19 2019, 12:25 PM · Origin-npm, Archive coverage
ardumont closed T2021: Re-implement pypi loader with package loader, a subtask of T1389: Implement a base "package" loader for package managers, as Resolved.
Nov 19 2019, 12:25 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont triaged T2098: Deploy package loaders as Normal priority.
Nov 19 2019, 12:25 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Oct 13 2019

ardumont changed the status of T2025: Re-implement debian loader with package loader mechanism, a subtask of T1389: Implement a base "package" loader for package managers, from Open to Work in Progress.
Oct 13 2019, 6:37 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Oct 5 2019

ardumont added a project to T1389: Implement a base "package" loader for package managers: Origin-Debian.
Oct 5 2019, 2:30 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont added a project to T1389: Implement a base "package" loader for package managers: Origin-CRAN.
Oct 5 2019, 2:29 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Oct 3 2019

ardumont changed the status of T2024: Re-implement deposit loader with package loader mechanism, a subtask of T1389: Implement a base "package" loader for package managers, from Open to Work in Progress.
Oct 3 2019, 6:03 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Oct 1 2019

ardumont closed T2017: package loader: Discuss revision metadata normalization, a subtask of T1389: Implement a base "package" loader for package managers, as Resolved.
Oct 1 2019, 6:46 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont added a subtask for T1389: Implement a base "package" loader for package managers: T2017: package loader: Discuss revision metadata normalization.
Oct 1 2019, 6:46 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont added a project to T1389: Implement a base "package" loader for package managers: Origin-GNU.
Oct 1 2019, 6:40 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont triaged T2026: Implement cran loader with package manager mechanism as Normal priority.
Oct 1 2019, 1:29 PM · Origin-CRAN, Archive coverage
ardumont added a comment to T1389: Implement a base "package" loader for package managers.

Current work is in the swh-loader-repository within the package-loader branch.

Oct 1 2019, 1:23 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont renamed T1389: Implement a base "package" loader for package managers from Implement a base loader for package managers to Implement a base "package" loader for package managers.
Oct 1 2019, 1:20 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont changed the status of T1389: Implement a base "package" loader for package managers from Open to Work in Progress.
Oct 1 2019, 1:20 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont changed the status of T2021: Re-implement pypi loader with package loader, a subtask of T1389: Implement a base "package" loader for package managers, from Open to Work in Progress.
Oct 1 2019, 1:19 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont changed the status of T2022: Re-implement npm loader with base loader, a subtask of T1389: Implement a base "package" loader for package managers, from Open to Work in Progress.
Oct 1 2019, 1:19 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage
ardumont changed the status of T2022: Re-implement npm loader with base loader from Open to Work in Progress.
Oct 1 2019, 1:19 PM · Origin-npm, Archive coverage
ardumont removed a project from T2022: Re-implement npm loader with base loader: Origin-Pypi.
Oct 1 2019, 1:18 PM · Origin-npm, Archive coverage
ardumont changed the status of T2023: Re-implement gnu loader with package loader from Open to Work in Progress.
Oct 1 2019, 1:14 PM · Origin-GNU, Archive coverage
ardumont triaged T2022: Re-implement npm loader with base loader as Normal priority.
Oct 1 2019, 1:12 PM · Origin-npm, Archive coverage
ardumont triaged T2021: Re-implement pypi loader with package loader as Normal priority.
Oct 1 2019, 1:11 PM · Origin-Pypi, Archive coverage

Sep 13 2019

nahimilega added a comment to T1389: Implement a base "package" loader for package managers.

@douardda We have a separate task(T1777) for rubygem lister. I guess we can add a separate column in the table by @anlambert showing status of the lister implementation and the tasks related to it.

Sep 13 2019, 9:03 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Sep 12 2019

douardda added a comment to T1389: Implement a base "package" loader for package managers.
Package managerPackages listing url Package metadata urlPackage source tarball url
RubygemsNo public api endpoint availablehttps://rubygems.org/api/v2/rubygems/[package]/versions/[version].jsonavailable in the medata
Sep 12 2019, 11:08 AM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Jul 7 2019

nahimilega added a revision to T1389: Implement a base "package" loader for package managers: D1694: swh.loader.package: Implement a method to prepare package visit.
Jul 7 2019, 10:10 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Jul 2 2019

nahimilega added a comment to T1389: Implement a base "package" loader for package managers.

Extending over the plan by @olasd. Here are some of my thoughts on the implementation of base loader.

Jul 2 2019, 12:46 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Jun 17 2019

olasd added a comment to T1389: Implement a base "package" loader for package managers.
In T1389#33215, @zack wrote:

Thanks @olasd, @ardumont, and @anlambert for this, it's a great plan and I like it a lot !

Just a few comments on the sidelines:

The lister will generate a one-shot task to load each package for the given repository, with the full information needed to do the data fetching.

This seemed clear from a different part of the description, but fwiw here I'm assuming the plan is to only load the version of the packages not already known/ingested in the past.

Jun 17 2019, 4:19 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Jun 12 2019

zack added a comment to T1389: Implement a base "package" loader for package managers.

Thanks @olasd, @ardumont, and @anlambert for this, it's a great plan and I like it a lot !

Jun 12 2019, 1:58 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Jun 7 2019

rdicosmo moved T1766: Write blog post about the npm ingestion from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.
Jun 7 2019, 6:14 PM · Origin-npm, Blog, Website, Unknown Object (Project)

Jun 3 2019

vlorentz moved T1766: Write blog post about the npm ingestion from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.
Jun 3 2019, 11:34 AM · Origin-npm, Blog, Website, Unknown Object (Project)
vlorentz moved T1766: Write blog post about the npm ingestion from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.
Jun 3 2019, 11:34 AM · Origin-npm, Blog, Website, Unknown Object (Project)

May 27 2019

anlambert triaged T1766: Write blog post about the npm ingestion as Normal priority.
May 27 2019, 3:39 PM · Origin-npm, Blog, Website, Unknown Object (Project)
anlambert closed T1378: Ingest npm into the Software Heritage archive (meta task) as Resolved.

npm is now archived continuously by Software Heritage (see for instance: https://archive.softwareheritage.org/browse/origin/https://www.npmjs.com/package/webpack/).

May 27 2019, 1:58 PM · Origin-npm, Archive coverage
anlambert closed T1761: Add npm logo to the archive coverage page, a subtask of T1378: Ingest npm into the Software Heritage archive (meta task), as Resolved.
May 27 2019, 1:41 PM · Origin-npm, Archive coverage
anlambert closed T1761: Add npm logo to the archive coverage page as Resolved by committing rDWAPPS26f94f6244a7: misc/coverage: Add npm origin type.
May 27 2019, 1:41 PM · Origin-npm, Archive coverage
anlambert triaged T1761: Add npm logo to the archive coverage page as Normal priority.
May 27 2019, 10:51 AM · Origin-npm, Archive coverage
anlambert closed T1629: Deploy the npm ingestion in production as Resolved.

It is done ! npm packages are now available in the archive content so closing this.

May 27 2019, 10:48 AM · Origin-npm

May 25 2019

zack added a comment to T1378: Ingest npm into the Software Heritage archive (meta task).

I think the only thing missing here is adding the NPM logo to the archive coverage page.

May 25 2019, 5:37 PM · Origin-npm, Archive coverage
zack added a comment to T1629: Deploy the npm ingestion in production.

I think this is now done, right @anlambert ?

May 25 2019, 5:37 PM · Origin-npm

May 20 2019

olasd updated subscribers of T1389: Implement a base "package" loader for package managers.

We've discussed a plausible plan for a "base package manager loader" with @ardumont and, to some extent, @anlambert.

May 20 2019, 6:03 PM · Origin-Debian, Origin-CRAN, Origin-GNU, Origin-npm, Origin-Pypi, Archive coverage

Apr 12 2019

anlambert closed T1644: Fix errors detected after a first loading batch as Resolved by committing rDLDNPMb234386f8ad5: npm.client: Handle parsing of package.json file non utf8 encoded.
Apr 12 2019, 10:48 AM · Origin-npm
anlambert closed T1644: Fix errors detected after a first loading batch, a subtask of T1629: Deploy the npm ingestion in production, as Resolved.
Apr 12 2019, 10:48 AM · Origin-npm

Apr 11 2019

anlambert triaged T1644: Fix errors detected after a first loading batch as Normal priority.
Apr 11 2019, 3:08 PM · Origin-npm
ardumont added a comment to T1629: Deploy the npm ingestion in production.

I've stopped the npm loader, and created a new dashboard to track the errors [1]

Apr 11 2019, 2:06 PM · Origin-npm
ardumont closed T1632: Update workers configuration for the npm ingestion, a subtask of T1629: Deploy the npm ingestion in production, as Resolved.
Apr 11 2019, 2:06 PM · Origin-npm
ardumont closed T1631: Add new scheduler task types for npm ingestion, a subtask of T1629: Deploy the npm ingestion in production, as Resolved.
Apr 11 2019, 1:46 PM · Origin-npm

Apr 10 2019

ardumont closed T1634: Create database for npm lister as Resolved.
Apr 10 2019, 3:57 PM · Npm Lister, Origin-npm
ardumont closed T1634: Create database for npm lister, a subtask of T1629: Deploy the npm ingestion in production, as Resolved.
Apr 10 2019, 3:57 PM · Origin-npm
ardumont added a comment to T1634: Create database for npm lister.
  1. Create role
Apr 10 2019, 3:57 PM · Npm Lister, Origin-npm
anlambert added a project to T1634: Create database for npm lister: Npm Lister.
Apr 10 2019, 10:53 AM · Npm Lister, Origin-npm
anlambert triaged T1634: Create database for npm lister as Normal priority.
Apr 10 2019, 10:53 AM · Npm Lister, Origin-npm

Apr 9 2019

olasd closed T1630: Add debian packaging for the npm loader, a subtask of T1629: Deploy the npm ingestion in production, as Resolved.
Apr 9 2019, 5:22 PM · Origin-npm
anlambert added a project to T1629: Deploy the npm ingestion in production: Origin-npm.
Apr 9 2019, 11:49 AM · Origin-npm

Apr 2 2019

anlambert closed T1379: npm loader, a subtask of T1378: Ingest npm into the Software Heritage archive (meta task), as Resolved.
Apr 2 2019, 3:38 PM · Origin-npm, Archive coverage
anlambert closed T1379: npm loader as Resolved by committing rDLDNPM35dd880e7725: swh.loader.npm.tasks: Add npm loader celery task.
Apr 2 2019, 3:38 PM · Npm loader, Archive coverage, Origin-npm
anlambert closed D1254: Implement a loader for npm.
Apr 2 2019, 3:38 PM · Npm loader, Origin-npm
swh-public-ci added a comment to D1254: Implement a loader for npm.

Build is green
See https://jenkins.softwareheritage.org/job/DLDNPM/job/tox/4/ for more details.

Apr 2 2019, 3:29 PM · Npm loader, Origin-npm
anlambert updated the diff for D1254: Implement a loader for npm.

Update: Address some vlorentz comments

Apr 2 2019, 3:27 PM · Npm loader, Origin-npm
anlambert added inline comments to D1254: Implement a loader for npm.
Apr 2 2019, 3:26 PM · Npm loader, Origin-npm
vlorentz added inline comments to D1254: Implement a loader for npm.
Apr 2 2019, 2:54 PM · Npm loader, Origin-npm
ardumont accepted D1254: Implement a loader for npm.
Apr 2 2019, 2:49 PM · Npm loader, Origin-npm
swh-public-ci added a comment to D1254: Implement a loader for npm.

Build is green
See https://jenkins.softwareheritage.org/job/DLDNPM/job/tox/3/ for more details.

Apr 2 2019, 2:44 PM · Npm loader, Origin-npm
anlambert updated the diff for D1254: Implement a loader for npm.
  • address ardumont comments
Apr 2 2019, 2:41 PM · Npm loader, Origin-npm
ardumont added inline comments to D1254: Implement a loader for npm.
Apr 2 2019, 2:20 PM · Npm loader, Origin-npm
anlambert added inline comments to D1254: Implement a loader for npm.
Apr 2 2019, 2:04 PM · Npm loader, Origin-npm
ardumont added a comment to D1254: Implement a loader for npm.

Build is green

Apr 2 2019, 1:43 PM · Npm loader, Origin-npm
swh-public-ci added a comment to D1254: Implement a loader for npm.

Build is green
See https://jenkins.softwareheritage.org/job/DLDNPM/job/tox/2/ for more details.

Apr 2 2019, 1:41 PM · Npm loader, Origin-npm
anlambert updated the diff for D1254: Implement a loader for npm.

Update: CI build should work now

Apr 2 2019, 1:39 PM · Npm loader, Origin-npm
ardumont accepted D1254: Implement a loader for npm.
Apr 2 2019, 12:32 PM · Npm loader, Origin-npm
swh-public-ci added a comment to D1254: Implement a loader for npm.

Build has FAILED

Apr 2 2019, 12:17 PM · Npm loader, Origin-npm
ardumont added inline comments to D1254: Implement a loader for npm.
Apr 2 2019, 12:17 PM · Npm loader, Origin-npm
douardda added inline comments to D1254: Implement a loader for npm.
Apr 2 2019, 12:06 PM · Npm loader, Origin-npm
anlambert updated the summary of D1254: Implement a loader for npm.
Apr 2 2019, 12:05 PM · Npm loader, Origin-npm
ardumont added inline comments to D1254: Implement a loader for npm.
Apr 2 2019, 11:56 AM · Npm loader, Origin-npm

Apr 1 2019

ardumont added a comment to D1254: Implement a loader for npm.

heh, just noticed you finalized it, i'll take a look tomorrow :)

Apr 1 2019, 11:09 PM · Npm loader, Origin-npm
anlambert updated the summary of D1254: Implement a loader for npm.
Apr 1 2019, 5:54 PM · Npm loader, Origin-npm
Harbormaster failed remote builds in B5000: Diff 4234 for D1254: Implement a loader for npm!
Apr 1 2019, 5:53 PM · Npm loader, Origin-npm
anlambert updated the diff for D1254: Implement a loader for npm.
  • finish code documentation and fill README.md
Apr 1 2019, 5:53 PM · Npm loader, Origin-npm

Mar 29 2019

Harbormaster failed remote builds in B4956: Diff 4193 for D1254: Implement a loader for npm!
Mar 29 2019, 5:12 PM · Npm loader, Origin-npm
anlambert updated the diff for D1254: Implement a loader for npm.
  • add more tests
Mar 29 2019, 5:11 PM · Npm loader, Origin-npm

Mar 28 2019

ardumont added a comment to D1254: Implement a loader for npm.

I have not read the tests yet but the runtime code sound good already (as a first feedback draft ;)

Mar 28 2019, 4:21 PM · Npm loader, Origin-npm

Mar 15 2019

anlambert updated the summary of D1254: Implement a loader for npm.
Mar 15 2019, 8:18 PM · Npm loader, Origin-npm
anlambert added projects to D1254: Implement a loader for npm: Origin-npm, Npm loader.
Mar 15 2019, 8:16 PM · Npm loader, Origin-npm