Page MenuHomeSoftware Heritage
Feed Advanced Search

Sep 9 2022

ardumont committed R260:da7245d8bf66: archive-staging: Deploy pubdev ingestion stack (authored by ardumont).
archive-staging: Deploy pubdev ingestion stack
Sep 9 2022, 9:54 AM
ardumont changed the status of T4517: staging: Deploy pubdev, a subtask of T4465: Ingest pub.dev (Dart, Flutter), from Open to Work in Progress.
Sep 9 2022, 9:51 AM · PubDev loader, PubDev lister, Archive coverage
ardumont changed the status of T4517: staging: Deploy pubdev from Open to Work in Progress.
Sep 9 2022, 9:51 AM · System administration, PubDev loader, PubDev lister, Archive coverage
ardumont added a project to T4517: staging: Deploy pubdev: System administration.
Sep 9 2022, 9:51 AM · System administration, PubDev loader, PubDev lister, Archive coverage
ardumont triaged T4517: staging: Deploy pubdev as Normal priority.
Sep 9 2022, 9:51 AM · System administration, PubDev loader, PubDev lister, Archive coverage
ardumont committed R260:e0ae21cf308f: staging/loader_package: Deploy new release v4.0 (authored by ardumont).
staging/loader_package: Deploy new release v4.0
Sep 9 2022, 9:47 AM
ardumont closed T4135: staging: Deploy graphql service, a subtask of T4103: Setup a staging environment for GraphQL APIs, as Resolved.
Sep 9 2022, 9:41 AM · GraphQL API
ardumont closed T4135: staging: Deploy graphql service as Resolved.
Sep 9 2022, 9:41 AM · System administration, GraphQL API
ardumont closed T4135: staging: Deploy graphql service, a subtask of T4134: Package the graphql service, as Resolved.
Sep 9 2022, 9:41 AM · System administration, GraphQL API

Sep 8 2022

ardumont accepted D8433: Add scratch disk to thanos host (for compaction).
Sep 8 2022, 7:43 PM
ardumont moved T4135: staging: Deploy graphql service from code-review/await-feedback/pause to deployed/landed/monitoring on the System administration board.
Sep 8 2022, 6:22 PM · System administration, GraphQL API
ardumont updated the task description for T4135: staging: Deploy graphql service.
Sep 8 2022, 6:21 PM · System administration, GraphQL API
ardumont updated the title for P1446 icinga not happy about missing 'host' from whatever icinga dude! be awesome to each other! to icinga not happy about missing 'host'.
Sep 8 2022, 6:08 PM
ardumont closed D8432: Add static check on the staging graphql instance 2/2.
Sep 8 2022, 6:08 PM
ardumont committed rSPSITEfa7e071f4f7f: Add static check on the staging graphql instance 2/2 (authored by ardumont).
Add static check on the staging graphql instance 2/2
Sep 8 2022, 6:07 PM
ardumont requested review of D8432: Add static check on the staging graphql instance 2/2.
Sep 8 2022, 6:07 PM
ardumont added a revision to T4135: staging: Deploy graphql service: D8432: Add static check on the staging graphql instance 2/2.
Sep 8 2022, 6:07 PM · System administration, GraphQL API
ardumont updated the title for P1446 icinga not happy about missing 'host' from whatever icinga dude! be awesome with each other! to whatever icinga dude! be awesome to each other!.
Sep 8 2022, 5:56 PM
ardumont edited P1446 icinga not happy about missing 'host'.
Sep 8 2022, 5:55 PM
ardumont created P1446 icinga not happy about missing 'host'.
Sep 8 2022, 5:55 PM
ardumont accepted D8431: swh-graph: configure the max heap allocated to the java backend.

w00t!

Sep 8 2022, 5:51 PM
ardumont closed D8429: Add static check on the staging graphql instance.
Sep 8 2022, 5:32 PM
ardumont committed rSPSITEdc47d24839ee: Add static check on the staging graphql instance (authored by ardumont).
Add static check on the staging graphql instance
Sep 8 2022, 5:32 PM
ardumont accepted D8427: Convert SWHID to str before passing to sentry_sdk.set_tag.
Sep 8 2022, 5:27 PM
ardumont updated the task description for T4135: staging: Deploy graphql service.
Sep 8 2022, 5:27 PM · System administration, GraphQL API
ardumont updated the diff for D8429: Add static check on the staging graphql instance.

Check some graphql specific content at the same time

Sep 8 2022, 5:26 PM
ardumont requested review of D8429: Add static check on the staging graphql instance.
Sep 8 2022, 5:24 PM
ardumont added a revision to T4135: staging: Deploy graphql service: D8429: Add static check on the staging graphql instance.
Sep 8 2022, 5:24 PM · System administration, GraphQL API
ardumont added a comment to D8422: tests/data/README.md: Use 'python3' executable instead of 'python'.

the venv is recommended, but stubborn people like me don't use it :)

Sep 8 2022, 5:16 PM
ardumont committed rSKCONF41c675db5a4c: archive-staging: Switch branch to development branch to test changes (authored by ardumont).
archive-staging: Switch branch to development branch to test changes
Sep 8 2022, 5:10 PM
ardumont committed R260:e326c3e94e81: archive-staging: Activate npm package loader (authored by ardumont).
archive-staging: Activate npm package loader
Sep 8 2022, 5:01 PM
ardumont committed R260:9a678ecd94f2: Use swh-loader-package image for package loaders (authored by ardumont).
Use swh-loader-package image for package loaders
Sep 8 2022, 4:57 PM
ardumont committed R259:223d96dc21eb: Add new swh-loader-package application (authored by ardumont).
Add new swh-loader-package application
Sep 8 2022, 4:56 PM
ardumont accepted D8412: webapp: Update configuration for next deployment.

one question inline.

Sep 8 2022, 4:51 PM
ardumont accepted D8392: Fix crash when RawExtrinsicMetadata target new origins.
Sep 8 2022, 4:47 PM
ardumont accepted D8416: github: Add support for 'topics'.
Sep 8 2022, 4:45 PM
ardumont updated the task description for T4135: staging: Deploy graphql service.
Sep 8 2022, 4:44 PM · System administration, GraphQL API
ardumont committed R260:a916c5c6b601: Use swh-loader-highpriority image for highpriority or package loader (authored by ardumont).
Use swh-loader-highpriority image for highpriority or package loader
Sep 8 2022, 4:12 PM
ardumont committed R260:779e0c46e4e6: Use swh-loader-highpriority image for highpriority or package loader (authored by ardumont).
Use swh-loader-highpriority image for highpriority or package loader
Sep 8 2022, 4:11 PM
ardumont committed rSKCONF8498a800ccaa: archive-staging: Switch branch to development branch to test changes (authored by ardumont).
archive-staging: Switch branch to development branch to test changes
Sep 8 2022, 4:06 PM
ardumont committed R260:90da5d2fee72: Use swh-loader-highpriority image for highpriority or package loader (authored by ardumont).
Use swh-loader-highpriority image for highpriority or package loader
Sep 8 2022, 4:05 PM
ardumont closed D8423: service-urls: Update public with graphql instance.
Sep 8 2022, 3:54 PM
ardumont committed rDDOC3108fa1e33fa: service-urls: Update public with graphql instance (authored by ardumont).
service-urls: Update public with graphql instance
Sep 8 2022, 3:54 PM
ardumont committed R259:9196f22cf16f: Add new swh-loader-highpriority application (authored by ardumont).
Add new swh-loader-highpriority application
Sep 8 2022, 3:49 PM
ardumont requested review of D8423: service-urls: Update public with graphql instance.
Sep 8 2022, 3:35 PM
ardumont updated the task description for T4135: staging: Deploy graphql service.
Sep 8 2022, 3:23 PM · System administration, GraphQL API
ardumont added a revision to T4135: staging: Deploy graphql service: D8423: service-urls: Update public with graphql instance.
Sep 8 2022, 3:22 PM · System administration, GraphQL API
ardumont closed T4399: OIN -> SWH deposit connection as Resolved.
Sep 8 2022, 3:09 PM · System administration, SWORD deposit
ardumont added a comment to T4399: OIN -> SWH deposit connection.

No feedback received but some deposits are showing up in the staging deposit db.
So i'm assuming all is fine.

Sep 8 2022, 3:08 PM · System administration, SWORD deposit
ardumont moved T4399: OIN -> SWH deposit connection from code-review/await-feedback/pause to deployed/landed/monitoring on the System administration board.
Sep 8 2022, 3:06 PM · System administration, SWORD deposit
ardumont closed T4415: Improve indexer workers performance as Resolved.
Sep 8 2022, 3:04 PM · System administration
ardumont added a comment to T4415: Improve indexer workers performance.

It seems to no longer be a problem now.
So closing.

Sep 8 2022, 3:04 PM · System administration
ardumont closed T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, as Resolved.
Sep 8 2022, 3:03 PM · Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont closed T4330: Deploy maven stack in production as Resolved.
Sep 8 2022, 3:03 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont closed T4511: production: Save Code Now requests for bazaar origins are not scheduled as Resolved.
Sep 8 2022, 3:03 PM · System administration, Save Code Now
ardumont accepted D8414: Remove unused 'get_traversal' method.

if you say so ¯\_(ツ)_/¯

Sep 8 2022, 2:57 PM
ardumont accepted D8417: Fix crash when indexing the same directory twice with non-deterministic order.
Sep 8 2022, 2:57 PM
ardumont accepted D8422: tests/data/README.md: Use 'python3' executable instead of 'python'.

'python' may be Python 2,...

Sep 8 2022, 2:55 PM
ardumont accepted D8401: user/listers: Add new listers in development to listers info page.
Sep 8 2022, 2:54 PM
ardumont accepted D8408: resolvers: Initialize backends in base classes for node and connection.
Sep 8 2022, 2:49 PM
ardumont accepted D8409: tests: Add some missing unknown object tests.
Sep 8 2022, 2:48 PM
ardumont moved T4330: Deploy maven stack in production from in-progress to deployed/landed/monitoring on the System administration board.
Sep 8 2022, 2:16 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 2:16 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a comment to T4330: Deploy maven stack in production.

Checks:

  • task has been scheduled by the scheduler runner process [1]
  • listing is being consumed by one worker [2]
  • 'maven' listed origins is steadily growing [3]
  • New 'maven' listed origins are getting scheduled for ingestion [4]
  • maven loaders are ingesting those [5]
Sep 8 2022, 2:16 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont closed D8421: production: Update lister to consume maven listing.
Sep 8 2022, 2:11 PM
ardumont committed rSPSITE11749c11a9cd: production: Update lister to consume maven listing (authored by ardumont).
production: Update lister to consume maven listing
Sep 8 2022, 2:11 PM
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 2:11 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont requested review of D8421: production: Update lister to consume maven listing.
Sep 8 2022, 2:11 PM
ardumont added a revision to T4330: Deploy maven stack in production: D8421: production: Update lister to consume maven listing.
Sep 8 2022, 2:11 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 2:06 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a comment to T4330: Deploy maven stack in production.

Schedule maven-central listing:

swhscheduler@saatchi:~$ curl -s https://repo1.maven.org/maven2/ | head -2
<!DOCTYPE html>
<html>
swhscheduler@saatchi:~$ curl -s https://maven-exporter.internal.softwareheritage.org/export-maven-central.fld | head -2
doc 0
  field 0
swhscheduler@saatchi:~$ curl -s http://saatchi.internal.softwareheritage.org:5008/
<html>
<head><title>Software Heritage scheduler RPC server</title></head>
<body>
<p>You have reached the
<a href="https://www.softwareheritage.org/">Software Heritage</a>
scheduler RPC server.<br />
See its
<a href="https://docs.softwareheritage.org/devel/swh-scheduler/">documentation
and API</a> for more information</p>
</body>
</html>swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/ \
>   task add list-maven-full \
>     url=https://repo1.maven.org/maven2/ \
>     index_url=https://maven-exporter.internal.softwareheritage.org/export-maven-central.fld
Created 1 tasks
Sep 8 2022, 2:04 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont updated the task description for T4330: Deploy maven stack in production.
Sep 8 2022, 1:47 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a comment to T4330: Deploy maven stack in production.

Finally, export is done on maven central [1], the fld is computed [2]...
And it's also exposed, hence reachable from lister worker nodes.

Sep 8 2022, 1:47 PM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont moved T4511: production: Save Code Now requests for bazaar origins are not scheduled from in-progress to deployed/landed/monitoring on the System administration board.
Sep 8 2022, 1:44 PM · System administration, Save Code Now
ardumont renamed T4511: production: Save Code Now requests for bazaar origins are not scheduled from Save Code Now requests for bazaar origins are not scheduled to production: Save Code Now requests for bazaar origins are not scheduled.
Sep 8 2022, 1:44 PM · System administration, Save Code Now
ardumont changed the status of T4511: production: Save Code Now requests for bazaar origins are not scheduled from Open to Work in Progress.
Sep 8 2022, 1:44 PM · System administration, Save Code Now
ardumont added a comment to T4511: production: Save Code Now requests for bazaar origins are not scheduled.

This is fixed now.

Sep 8 2022, 1:44 PM · System administration, Save Code Now
ardumont added projects to T4513: workers with multiple loaders do not send their report to sentry (high priority, future add-forge-now workers...): Developers, System administration.
Sep 8 2022, 1:42 PM · System administration, Developers
ardumont closed D8419: worker/instance: Allow sentry setup to be deactivated.
Sep 8 2022, 1:40 PM
ardumont committed rSPSITE7315d4f32329: worker/instance: Allow sentry setup to be deactivated (authored by ardumont).
worker/instance: Allow sentry setup to be deactivated
Sep 8 2022, 1:40 PM
ardumont requested review of D8419: worker/instance: Allow sentry setup to be deactivated.
Sep 8 2022, 1:39 PM
ardumont added a revision to T4513: workers with multiple loaders do not send their report to sentry (high priority, future add-forge-now workers...): D8419: worker/instance: Allow sentry setup to be deactivated.
Sep 8 2022, 1:39 PM · System administration, Developers
ardumont triaged T4513: workers with multiple loaders do not send their report to sentry (high priority, future add-forge-now workers...) as Normal priority.
Sep 8 2022, 1:31 PM · System administration, Developers
ardumont committed rSPSITE79ffa5465bb1: Make scheduler runner priority schedule save code now bzr origins (authored by ardumont).
Make scheduler runner priority schedule save code now bzr origins
Sep 8 2022, 1:29 PM
ardumont closed D8418: Make save code now workers consume bzr origins.
Sep 8 2022, 1:14 PM
ardumont committed rSPSITEdfd4db9c045a: Make save code now workers consume bzr origins (authored by ardumont).
Make save code now workers consume bzr origins
Sep 8 2022, 1:14 PM
ardumont added a revision to T4511: production: Save Code Now requests for bazaar origins are not scheduled: D8418: Make save code now workers consume bzr origins.
Sep 8 2022, 1:11 PM · System administration, Save Code Now
ardumont requested review of D8418: Make save code now workers consume bzr origins.
Sep 8 2022, 1:11 PM
ardumont committed R260:9f4ef7847c3f: staging/lister: Deploy new release v3.0 (authored by ardumont).
staging/lister: Deploy new release v3.0
Sep 8 2022, 11:45 AM
ardumont committed R259:3204e8456821: swh.lister: Generate new frozen requirements with release v3.0.0 (authored by ardumont).
swh.lister: Generate new frozen requirements with release v3.0.0
Sep 8 2022, 11:44 AM
ardumont changed the status of T4330: Deploy maven stack in production, a subtask of T1724: Maven Central repository support, from Open to Work in Progress.
Sep 8 2022, 11:18 AM · Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont changed the status of T4330: Deploy maven stack in production from Open to Work in Progress.
Sep 8 2022, 11:18 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont added a project to T4330: Deploy maven stack in production: System administration.
Sep 8 2022, 11:18 AM · System administration, Maven loader, Maven lister, GSoC 2019, Archive coverage
ardumont moved T4459: Deploy swh-indexer > v2.6 on staging then production from in-progress to Weekly backlog on the System administration board.
Sep 8 2022, 11:16 AM · Indexer, System administration
ardumont renamed T4459: Deploy swh-indexer > v2.6 on staging then production from Deploy swh-indexer v2.4 on production and staging to Deploy swh-indexer > v2.5 on production and staging.
Sep 8 2022, 11:16 AM · Indexer, System administration
ardumont moved T4144: Elastic worker infrastructure from Weekly backlog to in-progress on the System administration board.
Sep 8 2022, 11:16 AM · meta-task, System administration, Roadmap 2022
ardumont committed R260:f19493032390: archive-staging: Limit replica set to 2 per swh deployments (authored by ardumont).
archive-staging: Limit replica set to 2 per swh deployments
Sep 8 2022, 11:13 AM
ardumont committed rSKCONFd9be9b5bfd32: archive-staging: Switch back application branch to master (authored by ardumont).
archive-staging: Switch back application branch to master
Sep 8 2022, 11:10 AM
ardumont committed R260:90e4a70d64b4: archive/loader: Prepare configuration file within init-container (authored by ardumont).
archive/loader: Prepare configuration file within init-container
Sep 8 2022, 11:09 AM
ardumont committed R260:3b6644706c54: archive/loader: Use init-container to prepare configuration file (authored by ardumont).
archive/loader: Use init-container to prepare configuration file
Sep 8 2022, 10:57 AM