Page MenuHomeSoftware Heritage
Create Task
Maniphest T757

Memory leak in swh.storage.api.server
Closed, MigratedEdits Locked
Actions

Description

There's a memory leak somewere in the swh.storage.api.server stack.

We can mitigate it by setting gunicorn to reap workers after a set number of queries, but it's a pretty ugly thing to do.

Let's try to investigate and fix it...

Related Objects

Event Timeline

olasd created this task.Sep 11 2017, 12:15 PM
olasd mentioned this in rSPPROF8eab35e888cf: swh::deploy::storage: reap the workers every $foo requests.Sep 11 2017, 12:17 PM
ardumont renamed this task from Memroy leak in swh.storage.api.server to Memory leak in swh.storage.api.server.Sep 11 2017, 4:37 PM
olasd mentioned this in T760: swh api clients often fail with a BadStatusLine exception.Sep 13 2017, 12:33 PM
olasd mentioned this in rSPSITE8eab35e888cf: swh::deploy::storage: reap the workers every $foo requests.Jun 15 2018, 2:30 PM
olasd mentioned this in rSPSITE5160b1fe9a17: Reduce parallelism and max_requests on storage0 to reduce memory use.Jun 18 2018, 8:07 PM
olasd mentioned this in T1108: Excessive memory usage on storage0.euwest.azure.internal.softwareheritage.org.Jun 18 2018, 8:07 PM
zack raised the priority of this task from Normal to High.Jun 19 2018, 2:34 PM
zack added a subscriber: zack.Oct 12 2018, 10:00 AM

potentially useful related tooling: http://carsonip.me/posts/debugging-memory-usage-python-tracemalloc/

douardda added a subscriber: douardda.Jan 23 2020, 1:58 PM

Is this still "a thing"?

olasd closed this task as Wontfix.Jan 23 2020, 2:10 PM
olasd claimed this task.

Considering the age of the bug report and how many underlying libraries have been upgraded, we can reopen this when we notice it again.

gitlab-migration changed the task status from Wontfix to Migrated.Jan 8 2023, 4:21 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal