Page MenuHomeSoftware Heritage

cpan.loader - archive Perl modules from CPAN
Open, NormalPublic

Description

Hi SWH folks,

I would like to know if is there any plan or work-in-progress to preserve Perl modules from CPAN.org? CPAN[1] has a incredible archive containing 196,752 Perl modules and 42,031 distributions, also CPAN has a tradition to archive old versions of all modules through BackPan[2], which increase still more the total number of modules, but I'm not sure how much.

I'm interested in collaborate in this but I have no experience on SWH dev stack, also have no idea how much effort is to add a new loader on SWH, then this task here is a first step in it to know if there is already any kind of cpan.loader or if makes sense to have it on SWH.

[1] - https://www.cpan.org
[2] - http://backpan.perl.org

Thanks,
Joenio Marques da Costa
CorTexT platform / RISIS2.eu project

Event Timeline

joenio triaged this task as Wishlist priority.Dec 1 2020, 11:33 AM
joenio created this task.
joenio created this object in space S1 Public.

Hi SWH devs,

Any answer here?

I would like to know if a cpan.loader is something desired, if so I'm interested contributing working on it.

Thanks,
Joenio Marques da Costa
CorTexT platform / RISIS2.eu project

Hey, yes, we want to have one, but nobody is working it at the moment, and we rather have someone knowledgeable with that ecosystem to work on it. So, if you're interested, you're more than welcome to help there! (And thank you in advance.)

zack renamed this task from [feature request] cpan.loader - preserver Perl modules from CPAN to cpan.loader - preserver Perl modules from CPAN.Mar 30 2021, 8:22 AM
zack raised the priority of this task from Wishlist to Normal.

Thanks @zack for the info, I'll start learning the SWH dev stack following the instructions I found in the wiki[1].

[1] https://docs.softwareheritage.org/devel/

awesome, thanks @joenio ! you can also drop by our other devel communication channel if you want to discuss this in other ways: https://www.softwareheritage.org/community/developers/

Hi @joenio

I've just joined SWH as an ambassador and wondered how you are getting on with the cpan.loader? Or maybe metacpan.loader now?

Thanks,
Gavin.

zack renamed this task from cpan.loader - preserver Perl modules from CPAN to cpan.loader - archive Perl modules from CPAN.Apr 22 2022, 11:26 AM

CPANTS These indicators might be helpful in categorizing incoming data. So that the mining process is significantly easier

It's what metacpan-api uses.

Would we care for instant update mirroring for package downloads and then we can use a sort of post download extractor.

In T2833#83806, @ghenry wrote:

Hi @joenio

I've just joined SWH as an ambassador and wondered how you are getting on with the cpan.loader? Or maybe metacpan.loader now?

Thanks,
Gavin.

Hi @ghenry

I haven't started to develop nothing for this loader.

About the name cpan.loader or metacpan.loader, my bet is that it should continue as cpan.loader cause metacpan is just a search engine, and a new UI on top of CPAN but the repository is still named as CPAN.

bchauvet added a parent task: Unknown Object (Maniphest Task).Sep 2 2022, 11:00 AM