Page MenuHomeSoftware Heritage
Create Task
Maniphest T616

rate limiting for anonymous access to the API
Closed, MigratedEdits Locked
Actions

Description

To avoid denial of service on the underlying DB, and more generally to be able to limit resource consumption on our side, we want to be able to rate limit access to our JSON-based Web API for anonymous users.

This might be based on per-client policies (e.g., number of requests per unit of time per IP address) and/or global policies (e.g., total number of requests per unit of time, no matter who is submitting requests).

Given our current software stack, Flask Limiter might be the tool for the job (hat tip to @olasd for spotting it). FWIW, it doesn't seem to be Debian packaged yet.

Revisions and Commits

rSPSITE puppet-swh-site
rSPSITE448a917692a5 data/defaults: update webui rate limits after load testing (T652)

Related Objects
Search...

Event Timeline

zack created this task.Jan 4 2017, 8:30 PM
zack mentioned this in T614: Open up the Software Heritage API to the world.
zack added a project: Restricted Project.Jan 11 2017, 4:45 PM
zack moved this task from Restricted Project Column to Restricted Project Column on the Restricted Project board.
zack raised the priority of this task from Normal to High.Jan 12 2017, 11:39 AM
zack assigned this task to olasd.Jan 13 2017, 10:17 AM
zack mentioned this in T635: setup redis, as a service dependency for flask-limiter.Jan 18 2017, 1:20 PM
zack created subtask T635: setup redis, as a service dependency for flask-limiter.
olasd changed the status of subtask T635: setup redis, as a service dependency for flask-limiter from Open to Work in Progress.Jan 20 2017, 7:33 PM
olasd changed the task status from Open to Work in Progress.Jan 20 2017, 7:36 PM

All the relevant packages have been backported.

swh-web-ui now initializes Flask-Limiter in its setup.

The swh-web-ui configuration in Puppet has been adapted in consequence.

We just need to puppetize the Redis part of the configuration (the config has been modified live but not yet updated in puppet).

We also need to figure out what rate limits we want to give people.

olasd closed subtask T635: setup redis, as a service dependency for flask-limiter as Resolved.Jan 23 2017, 2:38 PM

The redis part of the configuration (URI and password) is now deployed through puppet.

zack created subtask T651: define rate limits for anonymous access to the API.Jan 30 2017, 2:05 PM
zack closed subtask T651: define rate limits for anonymous access to the API as Resolved.Feb 2 2017, 10:38 AM
olasd closed this task as Resolved by committing rSPSITE448a917692a5: data/defaults: update webui rate limits after load testing (T652).Feb 2 2017, 10:43 AM
olasd added a commit: rSPSITE448a917692a5: data/defaults: update webui rate limits after load testing (T652).
gitlab-migration changed the status of subtask T635: setup redis, as a service dependency for flask-limiter from Resolved to Migrated.Oct 19 2022, 5:52 PM
gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 4:20 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

gitlab-migration changed the status of subtask T651: define rate limits for anonymous access to the API from Resolved to Migrated.Jan 8 2023, 4:20 PM
About · Developer information · Legal