Page MenuHomeSoftware Heritage

rate limiting for anonymous access to the API
Closed, MigratedEdits Locked


To avoid denial of service on the underlying DB, and more generally to be able to limit resource consumption on our side, we want to be able to rate limit access to our JSON-based Web API for anonymous users.

This might be based on per-client policies (e.g., number of requests per unit of time per IP address) and/or global policies (e.g., total number of requests per unit of time, no matter who is submitting requests).

Given our current software stack, Flask Limiter might be the tool for the job (hat tip to @olasd for spotting it). FWIW, it doesn't seem to be Debian packaged yet.

Event Timeline

zack added a project: Restricted Project.Jan 11 2017, 4:45 PM
zack moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.
zack raised the priority of this task from Normal to High.Jan 12 2017, 11:39 AM
olasd changed the task status from Open to Work in Progress.Jan 20 2017, 7:36 PM

All the relevant packages have been backported.

swh-web-ui now initializes Flask-Limiter in its setup.

The swh-web-ui configuration in Puppet has been adapted in consequence.

We just need to puppetize the Redis part of the configuration (the config has been modified live but not yet updated in puppet).

We also need to figure out what rate limits we want to give people.

The redis part of the configuration (URI and password) is now deployed through puppet.