Page MenuHomeSoftware Heritage

Maven loaderFolder
ActivePublic

Members

  • This project does not have any members.
  • View All

Watchers

  • This project does not have any watchers.
  • View All

Details

Description

Loader for source packages from maven repositories

Recent Activity

Jan 8 2023

gitlab-migration closed T1724: Maven Central repository support as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:21 PM · Maven loader, Maven lister, GSoC 2019, Archive coverage
gitlab-migration changed the status of T3874: staging: Analyze result of the maven listing and ingestion, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Resolved to Migrated.
Jan 8 2023, 10:03 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T3874: staging: Analyze result of the maven listing and ingestion from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:03 PM · Maven loader, Maven lister, Archive coverage
gitlab-migration changed the status of T4318: Consider using jar command to extract jar archives from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:37 PM · Maven loader
gitlab-migration changed the status of T4232: Listers: Canonicalize listed github origins, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
Jan 8 2023, 4:36 PM · Maven loader, Maven lister, Archive coverage

Nov 3 2022

bchauvet added a parent task for T4330: Deploy maven stack in production: T4079: Extend archive coverage.
Nov 3 2022, 10:17 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage

Oct 19 2022

gitlab-migration changed the status of T4330: Deploy maven stack in production from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:07 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
gitlab-migration changed the status of T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, from Resolved to Migrated.
Oct 19 2022, 6:07 PM · Maven loader, Maven lister, GSoC 2019, Archive coverage
gitlab-migration changed the status of T4326: Archive the pom file additionally to the source folder, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Invalid to Migrated.
Oct 19 2022, 6:07 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4326: Archive the pom file additionally to the source folder from Invalid to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:07 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4215: staging: Deploy latest maven stack, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
Oct 19 2022, 6:06 PM · Maven loader, Maven lister, Archive coverage
gitlab-migration changed the status of T4215: staging: Deploy latest maven stack from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4143: staging: Deploy maven stack fixes, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
Oct 19 2022, 6:06 PM · Maven loader, Maven lister, Archive coverage
gitlab-migration changed the status of T4143: staging: Deploy maven stack fixes from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:06 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T4105: Push maven-index-exporter image to docker hub, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Resolved to Migrated.
Oct 19 2022, 6:06 PM · Maven loader, Maven lister, System administration, Archive coverage
gitlab-migration changed the status of T3746: staging: Deploy maven indexer/lister/loader, a subtask of T1724: Maven Central repository support, from Resolved to Migrated.
Oct 19 2022, 6:05 PM · Maven loader, Maven lister, GSoC 2019, Archive coverage
gitlab-migration changed the status of T3746: staging: Deploy maven indexer/lister/loader from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 6:05 PM · Maven loader, Maven lister, System administration, Archive coverage

Sep 8 2022

ardumont closed T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, as Resolved.
Sep 8 2022, 3:03 PM · Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont closed T4330: Deploy maven stack in production as Resolved.
Sep 8 2022, 3:03 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
vsellier added a comment to T4330: Deploy maven stack in production.

\o/ great

Sep 8 2022, 2:41 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont moved T4330: Deploy maven stack in production from in-progress to deployed/landed/monitoring on the System administration board.
Sep 8 2022, 2:16 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 2:16 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a comment to T4330: Deploy maven stack in production.

Checks:

  • task has been scheduled by the scheduler runner process [1]
  • listing is being consumed by one worker [2]
  • 'maven' listed origins is steadily growing [3]
  • New 'maven' listed origins are getting scheduled for ingestion [4]
  • maven loaders are ingesting those [5]
Sep 8 2022, 2:16 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 2:11 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a revision to T4330: Deploy maven stack in production: D8421: production: Update lister to consume maven listing.
Sep 8 2022, 2:11 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 2:06 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a comment to T4330: Deploy maven stack in production.

Schedule maven-central listing:

swhscheduler@saatchi:~$ curl -s https://repo1.maven.org/maven2/ | head -2
<!DOCTYPE html>
<html>
swhscheduler@saatchi:~$ curl -s https://maven-exporter.internal.softwareheritage.org/export-maven-central.fld | head -2
doc 0
  field 0
swhscheduler@saatchi:~$ curl -s http://saatchi.internal.softwareheritage.org:5008/
<html>
<head><title>Software Heritage scheduler RPC server</title></head>
<body>
<p>You have reached the
<a href="https://www.softwareheritage.org/">Software Heritage</a>
scheduler RPC server.<br />
See its
<a href="https://docs.softwareheritage.org/devel/swh-scheduler/">documentation
and API</a> for more information</p>
</body>
</html>swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/ \
>   task add list-maven-full \
>     url=https://repo1.maven.org/maven2/ \
>     index_url=https://maven-exporter.internal.softwareheritage.org/export-maven-central.fld
Created 1 tasks
Sep 8 2022, 2:04 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 1:47 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a comment to T4330: Deploy maven stack in production.

Finally, export is done on maven central [1], the fld is computed [2]...
And it's also exposed, hence reachable from lister worker nodes.

Sep 8 2022, 1:47 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont changed the status of T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, from Open to Work in Progress.
Sep 8 2022, 11:18 AM · Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont changed the status of T4330: Deploy maven stack in production from Open to Work in Progress.
Sep 8 2022, 11:18 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a project to T4330: Deploy maven stack in production: System administration.
Sep 8 2022, 11:18 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 10:30 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 10:19 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 10:05 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 9:57 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage

Sep 7 2022

ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 7 2022, 6:27 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage

Sep 6 2022

ardumont added a revision to T4330: Deploy maven stack in production: D8399: production: Provision new maven-exporter node.
Sep 6 2022, 2:56 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a revision to T4330: Deploy maven stack in production: D8397: Deploy maven-exporter production node.
Sep 6 2022, 11:50 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage

Jun 16 2022

ardumont updated the task description for T4330: Deploy maven stack in production.
Jun 16 2022, 11:13 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont triaged T4330: Deploy maven stack in production as Normal priority.
Jun 16 2022, 10:21 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage

Jun 15 2022

bchauvet closed T4326: Archive the pom file additionally to the source folder, a subtask of T3746: staging: Deploy maven indexer/lister/loader, as Invalid.
Jun 15 2022, 5:20 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet closed T4326: Archive the pom file additionally to the source folder as Invalid.
Jun 15 2022, 5:20 PM · Maven loader, Maven lister, System administration, Archive coverage
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.

Yesss! \o/

Jun 15 2022, 4:33 PM · Maven loader, Maven lister, System administration, Archive coverage
anlambert added a comment to T4326: Archive the pom file additionally to the source folder.

So in the end, the conclusion is that the loader already does the right thing so it's a noop, right?

Jun 15 2022, 4:26 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont renamed T4326: Archive the pom file additionally to the source folder from archive the pom file additionally to the source folder to Archive the pom file additionally to the source folder.
Jun 15 2022, 4:09 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont renamed T4326: Archive the pom file additionally to the source folder from archive the pom file additionnaly to the source folder to archive the pom file additionally to the source folder.
Jun 15 2022, 4:09 PM · Maven loader, Maven lister, System administration, Archive coverage
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.

Good news *can* happen, ahah! Thanks for notifying me.

Jun 15 2022, 3:42 PM · Maven loader, Maven lister, System administration, Archive coverage
ardumont updated subscribers of T4326: Archive the pom file additionally to the source folder.

To summarize, the initial intent was to adapt the jar loaded (as extracted directory) to append the pom.xml so we do not lose that reference.

Jun 15 2022, 3:33 PM · Maven loader, Maven lister, System administration, Archive coverage
bchauvet added a comment to T4326: Archive the pom file additionally to the source folder.

You're right boris, indeed it's already stored as extrinsic metadata, we hadn't checked properly :)
Thank you for your answer !

Jun 15 2022, 3:33 PM · Maven loader, Maven lister, System administration, Archive coverage