aka 'The Mirror'
Details
Details
Description
Jan 8 2023
Jan 8 2023
gitlab-migration closed T1607: Graph object count per object storage backend, a subtask of T1599: Analyze objstorage's Azure updateness, as Migrated.
gitlab-migration changed the status of T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch from Resolved to Migrated.
gitlab-migration changed the status of T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch, a subtask of T1608: Write all new objects to azure synchronously, from Resolved to Migrated.
gitlab-migration changed the status of T1608: Write all new objects to azure synchronously, a subtask of T1599: Analyze objstorage's Azure updateness, from Resolved to Migrated.
gitlab-migration changed the status of T1608: Write all new objects to azure synchronously from Resolved to Migrated.
gitlab-migration changed the status of T1606: Make swh-storage write to Kafka ahead of Postgresql from Duplicate to Migrated.
gitlab-migration changed the status of T1602: Analyze kakfa storage requirements from Resolved to Migrated.
gitlab-migration changed the status of T1601: Journal client of swh-storage mirrors from Resolved to Migrated.
gitlab-migration changed the status of T1600: Write a storage backend that writes to kafka from Resolved to Migrated.
gitlab-migration changed the status of T1599: Analyze objstorage's Azure updateness from Resolved to Migrated.
gitlab-migration changed the status of T1598: Add stasd probes in swh-storage from Resolved to Migrated.
gitlab-migration closed T1577: Compare/benchmark objstorage backends , a subtask of T1608: Write all new objects to azure synchronously, as Migrated.
Oct 19 2022
Oct 19 2022
gitlab-migration closed T1621: Add metrics to the currently deployed kafka cluster, a subtask of T1604: Improve kafka deployment, as Migrated.
gitlab-migration changed the status of T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance from Resolved to Migrated.
Jan 11 2021
Jan 11 2021
Sep 24 2020
Sep 24 2020
I guess this can be closed now
Jan 27 2020
Jan 27 2020
Jun 18 2019
Jun 18 2019
Jun 4 2019
Jun 4 2019
For the record, we do have a docker based deployment for this, but not a 'production' (aka based on debian packages) one. So I close this task for now. The task is now to document this (T1782).
May 14 2019
May 14 2019
So that took a few tries in puppet, but adding new brokers to the kafka deployment should now be seamless.
Apr 18 2019
Apr 18 2019
Apr 12 2019
Apr 12 2019
Closed by D1345
Apr 10 2019
Apr 10 2019
vlorentz moved T1607: Graph object count per object storage backend from Backlog to in progress on the Sprint 2019 03 board.
vlorentz moved T1621: Add metrics to the currently deployed kafka cluster from Backlog to in progress on the Sprint 2019 03 board.
vlorentz moved T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance from Backlog to done on the Sprint 2019 03 board.
vlorentz moved T1601: Journal client of swh-storage mirrors from Backlog to done on the Sprint 2019 03 board.
vlorentz moved T1603: kafka storage backfiller from Backlog to in progress on the Sprint 2019 03 board.
vlorentz moved T1600: Write a storage backend that writes to kafka from in progress to done on the Sprint 2019 03 board.
Apr 2 2019
Apr 2 2019
olasd changed the status of T1621: Add metrics to the currently deployed kafka cluster from Open to Work in Progress.
olasd triaged T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance as Normal priority.
zack renamed T1603: kafka storage backfiller from kafka storage backfill to kafka storage backfiller.
ardumont updated the task description for T1603: kafka storage backfiller.
Apr 1 2019
Apr 1 2019
douardda moved T1599: Analyze objstorage's Azure updateness from Backlog to in progress on the Sprint 2019 03 board.
Mar 29 2019
Mar 29 2019
ardumont added a comment to T1598: Add stasd probes in swh-storage.
Btw, i'll remove the @timed get_storage as this is just noisy on the graph ;)
(That concurs with what you said orally)
ardumont added a comment to T1598: Add stasd probes in swh-storage.
Some more granular probes inside the main storage implementation would be helpful, e.g. counting the actual number of objects inserted by each request.
Mar 28 2019
Mar 28 2019
olasd added a comment to T1598: Add stasd probes in swh-storage.
The current state of these probes has been deployed in production (and the workers restarted).
This is now deployed on the main storage. Let's see how it holds up.
olasd closed T1608: Write all new objects to azure synchronously, a subtask of T1599: Analyze objstorage's Azure updateness, as Resolved.