Page MenuHomeSoftware Heritage

Archive coverageFolder
ActivePublic

Members

  • This project does not have any members.
  • View All

Watchers

  • This project does not have any watchers.
  • View All

Details

Description

stuff related to extend the coverage of the Software Heritage archive

Recent Activity

Today

zack added a comment to T4346: Create SourceHut Lister.

This has now been discussed on the sourcehut mailing list and I took part in the conversation.

Mon, Aug 8, 11:14 AM · Archive coverage, Lister

Fri, Aug 5

ardumont updated the task description for T4104: Ingest crates.io (Rust).
Fri, Aug 5, 1:45 PM · Crates loader, Crates lister, Archive coverage
vlorentz added a comment to T4421: Prioritize archival from gitlab.com.

We discussed internally what to do with inactive repositories.
We reached a decision to move unused repos to object storage.
Once implemented, they will still be accessible but take a bit longer to access after a long period of inactivity.

Fri, Aug 5, 8:41 AM · Archive coverage, Origin-GitLab

Thu, Aug 4

olasd added a comment to T4421: Prioritize archival from gitlab.com.

Looks like there's many more repos that should be visitable but aren't:

Thu, Aug 4, 4:51 PM · Archive coverage, Origin-GitLab
ardumont added a comment to T1721: Implementation of Gogs Lister.

worth opening a dedicated forge issue

Done. T4423

open an upstream issue

Done. https://github.com/gogs/gogs/issues/7124

Thu, Aug 4, 4:02 PM · Archive coverage, Lister
ardumont raised the priority of T1721: Implementation of Gogs Lister from Low to Normal.
Thu, Aug 4, 3:59 PM · Archive coverage, Lister
ardumont raised the priority of T4423: Gogs pagination API breaks because of fatal repos from Low to Normal.
Thu, Aug 4, 3:58 PM · Lister, Archive coverage
olasd added a comment to T4421: Prioritize archival from gitlab.com.

updated query running:

Thu, Aug 4, 2:29 PM · Archive coverage, Origin-GitLab
olasd added a comment to T4421: Prioritize archival from gitlab.com.

As usual, I'm uneasy with the (general) idea of manually handling some repositories to resorb one bit of lag. This will only increase lag in another area that we will want to cover next. Rinse, repeat.

Thu, Aug 4, 2:18 PM · Archive coverage, Origin-GitLab
vlorentz added a comment to T4421: Prioritize archival from gitlab.com.

answer: 4755

Thu, Aug 4, 2:05 PM · Archive coverage, Origin-GitLab
vlorentz updated the task description for T4421: Prioritize archival from gitlab.com.
Thu, Aug 4, 10:38 AM · Archive coverage, Origin-GitLab
vlorentz added a comment to T4421: Prioritize archival from gitlab.com.

I am currently running a query to find how many origins are over one year overdue for a visit:

Thu, Aug 4, 10:29 AM · Archive coverage, Origin-GitLab
vlorentz triaged T4421: Prioritize archival from gitlab.com as Unbreak Now! priority.
Thu, Aug 4, 10:28 AM · Archive coverage, Origin-GitLab

Thu, Jul 28

ardumont added a project to T4411: Archive https://rsync.libreboot.org/: Archive coverage.
Thu, Jul 28, 5:49 PM · Archive coverage

Tue, Jul 19

vlorentz placed T1718: Implement a NuGet(.NET) lister up for grabs.
Tue, Jul 19, 11:24 PM · Archive coverage

Wed, Jul 13

vlorentz added a project to T4346: Create SourceHut Lister: Archive coverage.
Wed, Jul 13, 10:52 AM · Archive coverage, Lister

Jun 29 2022

franckbret added a comment to T4104: Ingest crates.io (Rust).

Will work on the incremental lister, and then document (not already done).

Jun 29 2022, 3:43 PM · Crates loader, Crates lister, Archive coverage
ardumont updated the task description for T4104: Ingest crates.io (Rust).
Jun 29 2022, 3:19 PM · Crates loader, Crates lister, Archive coverage
ardumont added a comment to T4104: Ingest crates.io (Rust).

What the next steps here?

Jun 29 2022, 3:19 PM · Crates loader, Crates lister, Archive coverage
ardumont updated the task description for T4104: Ingest crates.io (Rust).
Jun 29 2022, 3:15 PM · Crates loader, Crates lister, Archive coverage
ardumont updated the task description for T4104: Ingest crates.io (Rust).
Jun 29 2022, 3:15 PM · Crates loader, Crates lister, Archive coverage
franckbret added a comment to T4104: Ingest crates.io (Rust).

Hello,
The crates lister (stateless) and loader have landed.
I just solved some discovered issues while running lister and loader on the Docker env ( D8049 ).

Jun 29 2022, 3:03 PM · Crates loader, Crates lister, Archive coverage

Jun 21 2022

olasd changed the status of T4335: Archive repo.or.cz from Open to Work in Progress.

I've scheduled the archival of the 7377 repos in one of the leftover one-shot queues.

Jun 21 2022, 10:07 PM · Archive coverage
bchauvet moved T4233: Ingest Arch Linux from Restricted Project Column to Restricted Project Column on the Unknown Object (Project) board.
Jun 21 2022, 2:37 PM · Archive coverage, Unknown Object (Project)

Jun 19 2022

vlorentz updated the task description for T4335: Archive repo.or.cz.
Jun 19 2022, 9:25 AM · Archive coverage
vlorentz triaged T4335: Archive repo.or.cz as Unbreak Now! priority.
Jun 19 2022, 9:15 AM · Archive coverage

Jun 17 2022

ardumont updated the task description for T4233: Ingest Arch Linux.
Jun 17 2022, 9:53 AM · Archive coverage, Unknown Object (Project)

Jun 16 2022

ardumont updated the task description for T4233: Ingest Arch Linux.
Jun 16 2022, 2:47 PM · Archive coverage, Unknown Object (Project)
ardumont updated the task description for T4330: Deploy maven stack in production.
Jun 16 2022, 11:13 AM · Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont triaged T4330: Deploy maven stack in production as Normal priority.
Jun 16 2022, 10:21 AM · Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4233: Ingest Arch Linux.
Jun 16 2022, 10:02 AM · Archive coverage, Unknown Object (Project)
franckbret added a revision to T4233: Ingest Arch Linux: D7995: Arch Linux loader.
Jun 16 2022, 9:38 AM · Archive coverage, Unknown Object (Project)

Jun 15 2022

bchauvet closed T4326: Archive the pom file additionally to the source folder, a subtask of T3746: staging: Deploy maven indexer/lister/loader, as Invalid.
Jun 15 2022, 5:20 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet closed T4326: Archive the pom file additionally to the source folder as Invalid.
Jun 15 2022, 5:20 PM · Maven loader, Maven lister, System administration, Archive coverage
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.

Yesss! \o/

Jun 15 2022, 4:33 PM · Maven loader, Maven lister, System administration, Archive coverage
anlambert added a comment to T4326: Archive the pom file additionally to the source folder.

So in the end, the conclusion is that the loader already does the right thing so it's a noop, right?

Jun 15 2022, 4:26 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont renamed T4326: Archive the pom file additionally to the source folder from archive the pom file additionally to the source folder to Archive the pom file additionally to the source folder.
Jun 15 2022, 4:09 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont renamed T4326: Archive the pom file additionally to the source folder from archive the pom file additionnaly to the source folder to archive the pom file additionally to the source folder.
Jun 15 2022, 4:09 PM · Maven loader, Maven lister, System administration, Archive coverage
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.

Good news *can* happen, ahah! Thanks for notifying me.

Jun 15 2022, 3:42 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont updated subscribers of T4326: Archive the pom file additionally to the source folder.

To summarize, the initial intent was to adapt the jar loaded (as extracted directory) to append the pom.xml so we do not lose that reference.

Jun 15 2022, 3:33 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet added a comment to T4326: Archive the pom file additionally to the source folder.

You're right boris, indeed it's already stored as extrinsic metadata, we hadn't checked properly :)
Thank you for your answer !

Jun 15 2022, 3:33 PM · Maven loader, Maven lister, System administration, Archive coverage
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.

I'm not sure to understand the intent, as we already keep the pom in the extrinsic metadata (don't we?).
Double-checking in the SWH codebase, I believe you could build upon this: see [1] lines 166-180.

Jun 15 2022, 3:07 PM · Maven loader, Maven lister, System administration, Archive coverage
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.

Congrats on the work done! I think that downloading the pom file from the same folder is indeed the way to go.

Jun 15 2022, 2:54 PM · Maven loader, Maven lister, System administration, Archive coverage
anlambert added a comment to T4326: Archive the pom file additionally to the source folder.

I think the simplest way to get the pom file associated to a specific release of a maven package is to download it from the folder where we can find the source jar.

Jun 15 2022, 1:48 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont updated the task description for T4233: Ingest Arch Linux.
Jun 15 2022, 10:47 AM · Archive coverage, Unknown Object (Project)

Jun 13 2022

bchauvet updated the task description for T4326: Archive the pom file additionally to the source folder.
Jun 13 2022, 2:21 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet added a comment to T4326: Archive the pom file additionally to the source folder.
Jun 13 2022, 2:16 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet updated the task description for T4326: Archive the pom file additionally to the source folder.
Jun 13 2022, 1:36 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet updated the task description for T4326: Archive the pom file additionally to the source folder.
Jun 13 2022, 1:30 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet added a comment to T4326: Archive the pom file additionally to the source folder.

in a source.jar, the pom is not inculded by default but can be if specified :

Jun 13 2022, 1:27 PM · Maven loader, Maven lister, System administration, Archive coverage