Page MenuHomeSoftware Heritage

Write all new objects to azure synchronously
Closed, ResolvedPublic

Description

To fill the Rocquencourt / Azure gap, we need to make sure all new objects are written to azure synchronously.

The previous experimentation of this setup failed because the performance hit was gigantic. We've decided to

  • deploy the multiplexing object storage backend
  • investigate (and mitigate) the multiplexing performance issues (by deploying the statsd probes that are currently WIP, and tracing the actual behavior of the code)

Event Timeline

olasd created this task.Mar 26 2019, 6:57 PM
olasd triaged this task as High priority.
olasd closed this task as Resolved.Mar 28 2019, 6:10 PM

This is now deployed on the main storage. Let's see how it holds up.