Lister for maven repositories
Details
Details
Description
Jan 8 2023
Jan 8 2023
gitlab-migration changed the status of T3874: staging: Analyze result of the maven listing and ingestion, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Resolved to Migrated.
gitlab-migration changed the status of T3874: staging: Analyze result of the maven listing and ingestion from Resolved to Migrated.
gitlab-migration changed the status of T4232: Listers: Canonicalize listed github origins, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
Nov 3 2022
Nov 3 2022
bchauvet added a parent task for T4330: Deploy maven stack in production: T4079: Extend archive coverage.
Oct 19 2022
Oct 19 2022
gitlab-migration changed the status of T4330: Deploy maven stack in production from Resolved to Migrated.
gitlab-migration changed the status of T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, from Resolved to Migrated.
gitlab-migration changed the status of T4326: Archive the pom file additionally to the source folder, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Invalid to Migrated.
gitlab-migration changed the status of T4326: Archive the pom file additionally to the source folder from Invalid to Migrated.
gitlab-migration changed the status of T4215: staging: Deploy latest maven stack, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
gitlab-migration changed the status of T4215: staging: Deploy latest maven stack from Resolved to Migrated.
gitlab-migration changed the status of T4143: staging: Deploy maven stack fixes, a subtask of T3874: staging: Analyze result of the maven listing and ingestion, from Resolved to Migrated.
gitlab-migration changed the status of T4143: staging: Deploy maven stack fixes from Resolved to Migrated.
gitlab-migration changed the status of T4105: Push maven-index-exporter image to docker hub from Resolved to Migrated.
gitlab-migration changed the status of T4105: Push maven-index-exporter image to docker hub, a subtask of T3746: staging: Deploy maven indexer/lister/loader, from Resolved to Migrated.
gitlab-migration changed the status of T3746: staging: Deploy maven indexer/lister/loader, a subtask of T1724: Maven Central repository support, from Resolved to Migrated.
gitlab-migration changed the status of T3746: staging: Deploy maven indexer/lister/loader from Resolved to Migrated.
Sep 8 2022
Sep 8 2022
ardumont closed T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, as Resolved.
vsellier added a comment to T4330: Deploy maven stack in production.
\o/ great
ardumont moved T4330: Deploy maven stack in production from in-progress to deployed/landed/monitoring on the System administration board.
ardumont updated the task description for T4330: Deploy maven stack in production.
ardumont added a comment to T4330: Deploy maven stack in production.
Checks:
- task has been scheduled by the scheduler runner process [1]
- listing is being consumed by one worker [2]
- 'maven' listed origins is steadily growing [3]
- New 'maven' listed origins are getting scheduled for ingestion [4]
- maven loaders are ingesting those [5]
ardumont updated the task description for T4330: Deploy maven stack in production.
ardumont updated the task description for T4330: Deploy maven stack in production.
ardumont added a comment to T4330: Deploy maven stack in production.
Schedule maven-central listing:
swhscheduler@saatchi:~$ curl -s https://repo1.maven.org/maven2/ | head -2 <!DOCTYPE html> <html> swhscheduler@saatchi:~$ curl -s https://maven-exporter.internal.softwareheritage.org/export-maven-central.fld | head -2 doc 0 field 0 swhscheduler@saatchi:~$ curl -s http://saatchi.internal.softwareheritage.org:5008/ <html> <head><title>Software Heritage scheduler RPC server</title></head> <body> <p>You have reached the <a href="https://www.softwareheritage.org/">Software Heritage</a> scheduler RPC server.<br /> See its <a href="https://docs.softwareheritage.org/devel/swh-scheduler/">documentation and API</a> for more information</p> </body> </html>swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/ \ > task add list-maven-full \ > url=https://repo1.maven.org/maven2/ \ > index_url=https://maven-exporter.internal.softwareheritage.org/export-maven-central.fld Created 1 tasks
ardumont updated the task description for T4330: Deploy maven stack in production.
ardumont added a comment to T4330: Deploy maven stack in production.
Finally, export is done on maven central [1], the fld is computed [2]...
And it's also exposed, hence reachable from lister worker nodes.
ardumont changed the status of T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, from Open to Work in Progress.
ardumont changed the status of T4330: Deploy maven stack in production from Open to Work in Progress.
ardumont updated the task description for T4330: Deploy maven stack in production.
ardumont updated the task description for T4330: Deploy maven stack in production.
ardumont updated the task description for T4330: Deploy maven stack in production.
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 7 2022
Sep 7 2022
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 6 2022
Sep 6 2022
ardumont added a revision to T4330: Deploy maven stack in production: D8397: Deploy maven-exporter production node.
Jun 16 2022
Jun 16 2022
ardumont updated the task description for T4330: Deploy maven stack in production.
Jun 15 2022
Jun 15 2022
bchauvet closed T4326: Archive the pom file additionally to the source folder, a subtask of T3746: staging: Deploy maven indexer/lister/loader, as Invalid.
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.
Yesss! \o/
anlambert added a comment to T4326: Archive the pom file additionally to the source folder.
So in the end, the conclusion is that the loader already does the right thing so it's a noop, right?
ardumont renamed T4326: Archive the pom file additionally to the source folder from archive the pom file additionally to the source folder to Archive the pom file additionally to the source folder.
ardumont renamed T4326: Archive the pom file additionally to the source folder from archive the pom file additionnaly to the source folder to archive the pom file additionally to the source folder.
borisbaldassari added a comment to T4326: Archive the pom file additionally to the source folder.
Good news *can* happen, ahah! Thanks for notifying me.
ardumont updated subscribers of T4326: Archive the pom file additionally to the source folder.
To summarize, the initial intent was to adapt the jar loaded (as extracted directory) to append the pom.xml so we do not lose that reference.
bchauvet added a comment to T4326: Archive the pom file additionally to the source folder.
You're right boris, indeed it's already stored as extrinsic metadata, we hadn't checked properly :)
Thank you for your answer !