Page MenuHomeSoftware Heritage

Run swh-graph with gunicorn to support multiple/parallel requests
Open, HighPublic

Description

Currently it's ran on granet with the CLI: /opt/swhgraph_venv/bin/python3 /opt/swhgraph_venv/bin/swh graph rpc-serve -g /dev/shm/swh-graph/default/graph

This means it uses a single process, which is currently the bottleneck when processing a dozen queries in parallel.

The app path is a bit different from other packages, it's swh.graph.server.app:make_app_from_configfile()

Event Timeline

vlorentz triaged this task as Normal priority.Thu, Sep 30, 2:27 PM
vlorentz created this task.
vlorentz updated the task description. (Show Details)
vlorentz updated the task description. (Show Details)
vlorentz updated the task description. (Show Details)
vlorentz lowered the priority of this task from Normal to Low.EditedThu, Sep 30, 3:04 PM

Hmm actually this might be harder than just using gunicorn, because the java subprocess needs to be shared between workers, hmm...

I see there are some unapplied performance improvements in swh-graph, let's try that first.

zack renamed this task from Run swh-graph with gunicorn to Run swh-graph with gunicorn to support multiple/parallel requests.Sat, Oct 2, 8:01 AM
zack raised the priority of this task from Low to Normal.
zack raised the priority of this task from Normal to High.Sat, Oct 2, 8:06 AM