Feed Advanced Search

Advanced Search
Use Results
Edit Query
Hide Query

Jan 8 2023

gitlab-migration closed T1607: Graph object count per object storage backend, a subtask of T1599: Analyze objstorage's Azure updateness, as Migrated.

Jan 8 2023, 10:21 PM · Sprint 2019 03

gitlab-migration closed T1607: Graph object count per object storage backend as Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 10:21 PM · Metrics/monitoring, Sprint 2019 03

gitlab-migration changed the status of T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch, a subtask of T1608: Write all new objects to azure synchronously, from Resolved to Migrated.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1608: Write all new objects to azure synchronously, a subtask of T1599: Analyze objstorage's Azure updateness, from Resolved to Migrated.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1608: Write all new objects to azure synchronously from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1606: Make swh-storage write to Kafka ahead of Postgresql from Duplicate to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1602: Analyze kakfa storage requirements from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1601: Journal client of swh-storage mirrors from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1600: Write a storage backend that writes to kafka from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1599: Analyze objstorage's Azure updateness from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration changed the status of T1598: Add stasd probes in swh-storage from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 9:59 PM · Sprint 2019 03

gitlab-migration closed T1577: Compare/benchmark objstorage backends , a subtask of T1608: Write all new objects to azure synchronously, as Migrated.

Jan 8 2023, 5:00 PM · Sprint 2019 03

gitlab-migration changed the status of T1603: kafka storage backfiller from Resolved to Migrated.

This task has been migrated to GitLab.

Jan 8 2023, 4:26 PM · Journal, Sprint 2019 03

Oct 19 2022

gitlab-migration closed T1621: Add metrics to the currently deployed kafka cluster, a subtask of T1604: Improve kafka deployment, as Migrated.

Oct 19 2022, 5:55 PM · System administration, Sprint 2019 03

gitlab-migration closed T1621: Add metrics to the currently deployed kafka cluster as Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:55 PM · System administration, Metrics/monitoring, Sprint 2019 03

gitlab-migration changed the status of T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:55 PM · System administration, Sprint 2019 03

gitlab-migration changed the status of T1604: Improve kafka deployment from Resolved to Migrated.

This task has been migrated to GitLab.

Oct 19 2022, 5:55 PM · System administration, Sprint 2019 03

Jan 11 2021

vlorentz set the icon for Sprint 2019 03 to Goal.

Jan 11 2021, 1:46 PM

Sep 24 2020

douardda closed T1601: Journal client of swh-storage mirrors as Resolved.

I guess this can be closed now

Sep 24 2020, 2:42 PM · Sprint 2019 03

Jan 27 2020

vlorentz closed T1602: Analyze kakfa storage requirements as Resolved.

Jan 27 2020, 4:32 PM · Sprint 2019 03

Jun 18 2019

olasd closed T1600: Write a storage backend that writes to kafka as Resolved.

Jun 18 2019, 2:54 PM · Sprint 2019 03

vlorentz reassigned T1601: Journal client of swh-storage mirrors from vlorentz to douardda.

Jun 18 2019, 1:26 PM · Sprint 2019 03

Jun 4 2019

douardda closed T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance as Resolved.

For the record, we do have a docker based deployment for this, but not a 'production' (aka based on debian packages) one. So I close this task for now. The task is now to document this (T1782).

Jun 4 2019, 10:19 AM · System administration, Sprint 2019 03

May 14 2019

olasd closed T1604: Improve kafka deployment as Resolved.

So that took a few tries in puppet, but adding new brokers to the kafka deployment should now be seamless.

May 14 2019, 7:49 PM · System administration, Sprint 2019 03

Apr 18 2019

douardda closed T1599: Analyze objstorage's Azure updateness as Resolved.

Apr 18 2019, 10:35 AM · Sprint 2019 03

Apr 12 2019

ardumont closed T1603: kafka storage backfiller as Resolved.

Closed by D1345

Apr 12 2019, 3:50 PM · Journal, Sprint 2019 03

ardumont closed T1598: Add stasd probes in swh-storage as Resolved.

Apr 12 2019, 3:49 PM · Sprint 2019 03

Apr 10 2019

vlorentz moved T1607: Graph object count per object storage backend from Backlog to in progress on the Sprint 2019 03 board.

Apr 10 2019, 10:52 AM · Metrics/monitoring, Sprint 2019 03

vlorentz moved T1621: Add metrics to the currently deployed kafka cluster from Backlog to in progress on the Sprint 2019 03 board.

Apr 10 2019, 10:52 AM · System administration, Metrics/monitoring, Sprint 2019 03

vlorentz moved T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance from Backlog to done on the Sprint 2019 03 board.

Apr 10 2019, 10:51 AM · System administration, Sprint 2019 03

vlorentz moved T1601: Journal client of swh-storage mirrors from Backlog to done on the Sprint 2019 03 board.

Apr 10 2019, 10:51 AM · Sprint 2019 03

vlorentz claimed T1601: Journal client of swh-storage mirrors.

Apr 10 2019, 10:51 AM · Sprint 2019 03

vlorentz moved T1603: kafka storage backfiller from Backlog to in progress on the Sprint 2019 03 board.

Apr 10 2019, 10:50 AM · Journal, Sprint 2019 03

vlorentz moved T1600: Write a storage backend that writes to kafka from in progress to done on the Sprint 2019 03 board.

Apr 10 2019, 10:50 AM · Sprint 2019 03

Apr 2 2019

zack added a project to T1603: kafka storage backfiller: Journal.

Apr 2 2019, 2:44 PM · Journal, Sprint 2019 03

zack added a project to T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance: System administration.

Apr 2 2019, 2:43 PM · System administration, Sprint 2019 03

zack added a project to T1604: Improve kafka deployment: System administration.

Apr 2 2019, 2:43 PM · System administration, Sprint 2019 03

zack added projects to T1621: Add metrics to the currently deployed kafka cluster: Metrics/monitoring, System administration.

Apr 2 2019, 2:43 PM · System administration, Metrics/monitoring, Sprint 2019 03

olasd changed the status of T1621: Add metrics to the currently deployed kafka cluster from Open to Work in Progress.

Apr 2 2019, 12:56 PM · System administration, Metrics/monitoring, Sprint 2019 03

olasd triaged T1620: Sprint deliverable: deploy a kafka-in-the-loop mirror instance as Normal priority.

Apr 2 2019, 12:53 PM · System administration, Sprint 2019 03

zack renamed T1603: kafka storage backfiller from kafka storage backfill to kafka storage backfiller.

Apr 2 2019, 12:37 PM · Journal, Sprint 2019 03

ardumont claimed T1603: kafka storage backfiller.

Apr 2 2019, 12:11 PM · Journal, Sprint 2019 03

ardumont updated the task description for T1603: kafka storage backfiller.

Apr 2 2019, 12:10 PM · Journal, Sprint 2019 03

Apr 1 2019

douardda moved T1599: Analyze objstorage's Azure updateness from Backlog to in progress on the Sprint 2019 03 board.

Apr 1 2019, 11:54 AM · Sprint 2019 03

Mar 29 2019

ardumont added a comment to T1598: Add stasd probes in swh-storage.

Btw, i'll remove the @timed get_storage as this is just noisy on the graph ;)
(That concurs with what you said orally)

Mar 29 2019, 10:56 AM · Sprint 2019 03

ardumont added a comment to T1598: Add stasd probes in swh-storage.

Some more granular probes inside the main storage implementation would be helpful, e.g. counting the actual number of objects inserted by each request.

Mar 29 2019, 10:47 AM · Sprint 2019 03

Mar 28 2019

olasd added a comment to T1598: Add stasd probes in swh-storage.

The current state of these probes has been deployed in production (and the workers restarted).

Mar 28 2019, 6:38 PM · Sprint 2019 03

olasd closed T1608: Write all new objects to azure synchronously as Resolved.

This is now deployed on the main storage. Let's see how it holds up.

Mar 28 2019, 6:10 PM · Sprint 2019 03

olasd closed T1608: Write all new objects to azure synchronously, a subtask of T1599: Analyze objstorage's Azure updateness, as Resolved.

Mar 28 2019, 6:10 PM · Sprint 2019 03

Mar 27 2019

olasd closed T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch, a subtask of T1608: Write all new objects to azure synchronously, as Resolved.

Mar 27 2019, 12:14 PM · Sprint 2019 03

olasd closed T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch as Resolved by committing rSPSITE30b0c3819523: Enable some backported packages by default system-wide.

Mar 27 2019, 12:14 PM · Sprint 2019 03

olasd added a comment to T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch.

In T1609#29868, @anlambert wrote:

@olasd, urllib3 1.124.1 is available on stretch-backports [1], including the commit you mentioned.

[1] https://packages.debian.org/stretch-backports/python-urllib3

Mar 27 2019, 10:55 AM · Sprint 2019 03

anlambert added a comment to T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch.

@olasd, urllib3 1.24.1 is available on stretch-backports [1], including the commit you mentioned.

Mar 27 2019, 10:55 AM · Sprint 2019 03

Mar 26 2019

olasd triaged T1609: Mitigate azure objstorage http(s) pipelining issues by backporting urllib3 patch as High priority.

Mar 26 2019, 7:05 PM · Sprint 2019 03

olasd added a subtask for T1608: Write all new objects to azure synchronously: T1577: Compare/benchmark objstorage backends .

Mar 26 2019, 6:58 PM · Sprint 2019 03

olasd triaged T1608: Write all new objects to azure synchronously as High priority.

Mar 26 2019, 6:57 PM · Sprint 2019 03

olasd changed the status of T1607: Graph object count per object storage backend, a subtask of T1599: Analyze objstorage's Azure updateness, from Open to Work in Progress.

Mar 26 2019, 6:52 PM · Sprint 2019 03

olasd changed the status of T1607: Graph object count per object storage backend from Open to Work in Progress.

https://grafana.softwareheritage.org/d/jScG7g6mk/objstorage-object-counts shows the data that we're currently able to collect.

Mar 26 2019, 6:52 PM · Metrics/monitoring, Sprint 2019 03

vlorentz added a revision to T1600: Write a storage backend that writes to kafka: D1294: Add a new JournalWriter interface, which is notified by swh-storage before writing to pgsql..

Mar 26 2019, 5:10 PM · Sprint 2019 03

olasd triaged T1607: Graph object count per object storage backend as High priority.

Mar 26 2019, 4:31 PM · Metrics/monitoring, Sprint 2019 03

vlorentz claimed T1600: Write a storage backend that writes to kafka.

Mar 26 2019, 3:15 PM · Sprint 2019 03

vlorentz moved T1600: Write a storage backend that writes to kafka from Backlog to in progress on the Sprint 2019 03 board.

Mar 26 2019, 3:14 PM · Sprint 2019 03

vlorentz merged T1606: Make swh-storage write to Kafka ahead of Postgresql into T1600: Write a storage backend that writes to kafka.

Mar 26 2019, 3:14 PM · Sprint 2019 03

vlorentz merged task T1606: Make swh-storage write to Kafka ahead of Postgresql into T1600: Write a storage backend that writes to kafka.

Mar 26 2019, 3:14 PM · Sprint 2019 03

vlorentz moved T1606: Make swh-storage write to Kafka ahead of Postgresql from Backlog to in progress on the Sprint 2019 03 board.

Mar 26 2019, 3:13 PM · Sprint 2019 03

vlorentz claimed T1606: Make swh-storage write to Kafka ahead of Postgresql.

Mar 26 2019, 3:13 PM · Sprint 2019 03

vlorentz triaged T1606: Make swh-storage write to Kafka ahead of Postgresql as Normal priority.

Mar 26 2019, 3:13 PM · Sprint 2019 03

ardumont moved T1598: Add stasd probes in swh-storage from Backlog to in progress on the Sprint 2019 03 board.

Mar 26 2019, 2:02 PM · Sprint 2019 03

ardumont changed the status of T1598: Add stasd probes in swh-storage from Open to Work in Progress.

Mar 26 2019, 2:02 PM · Sprint 2019 03

Mar 25 2019

olasd added a comment to T1600: Write a storage backend that writes to kafka.

The intent is to start feeding kafka with some "production-grade" data, in a more reliable fashion that the current PostgreSQL notify mechanism. This will also help us get a better sense of the shape of the data, to make the analysis in T1602 more relevant.

Mar 25 2019, 4:44 PM · Sprint 2019 03

zack triaged T1602: Analyze kakfa storage requirements as Normal priority.

Mar 25 2019, 3:07 PM · Sprint 2019 03

zack triaged T1599: Analyze objstorage's Azure updateness as Normal priority.

Mar 25 2019, 3:07 PM · Sprint 2019 03

zack triaged T1603: kafka storage backfiller as Normal priority.

Mar 25 2019, 3:07 PM · Journal, Sprint 2019 03

zack triaged T1601: Journal client of swh-storage mirrors as Normal priority.

Mar 25 2019, 3:07 PM · Sprint 2019 03

zack triaged T1600: Write a storage backend that writes to kafka as Normal priority.

Mar 25 2019, 3:07 PM · Sprint 2019 03

zack triaged T1604: Improve kafka deployment as Normal priority.

Mar 25 2019, 3:07 PM · System administration, Sprint 2019 03

douardda created T1604: Improve kafka deployment.

Mar 25 2019, 11:32 AM · System administration, Sprint 2019 03

douardda created T1603: kafka storage backfiller.

Mar 25 2019, 11:31 AM · Journal, Sprint 2019 03

douardda created T1602: Analyze kakfa storage requirements.

Mar 25 2019, 11:31 AM · Sprint 2019 03

douardda added a comment to T1601: Journal client of swh-storage mirrors.

conflict policy -> update

Mar 25 2019, 11:30 AM · Sprint 2019 03

douardda created T1601: Journal client of swh-storage mirrors.

Mar 25 2019, 11:29 AM · Sprint 2019 03

douardda created T1600: Write a storage backend that writes to kafka.

Mar 25 2019, 11:28 AM · Sprint 2019 03

douardda created T1599: Analyze objstorage's Azure updateness.

Mar 25 2019, 11:27 AM · Sprint 2019 03

douardda triaged T1598: Add stasd probes in swh-storage as Normal priority.

Mar 25 2019, 11:26 AM · Sprint 2019 03

douardda renamed Sprint 2019 03 from Spint 2019 03 to Sprint 2019 03.

Mar 25 2019, 11:25 AM

douardda changed the edit policy for Sprint 2019 03.

Mar 25 2019, 10:55 AM