We now have a public-facing method in the Web API that accept POST data (/known).
We should have a catch-all/last resort limitation in the size of incoming POST request for all API methods to avoid abuses.
That specific endpoint will accept requests of the order of few tens of KiB (50 KiB in the current proposal for T2276). I'm guessing a significantly larger varnish limit, e.g., 1 MiB (?), would be enough to avoid having to fiddle with it too often and still prevent significant abuses.
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Migrated | gitlab-migration | T2276 Web API: /known: add a length limit to the list of accepted PID | ||
| Migrated | gitlab-migration | T2277 varnish: limit maximum size of incoming POST requests for Web API |
According the django documentation, the DATA_UPLOAD_MAX_MEMORY_SIZE variable should do the job.
By default, the variable is set to 2.5 MB. From my experience, 1 MB is sufficient (for example, it's also the limit for some JS library).
So, it looks like:
swh-web/swh/web/settings/common.py (l. 159)
...
REST_FRAMEWORK: Dict[str, Any] = {
"DATA_UPLOAD_MAX_MEMORY_SIZE": 1048576,
"DEFAULT_RENDERER_CLASSES": (
...My bad, the Web API is running behind HTTP reverse proxy (varnish). I'll propose a more appropriate solution.
After investigation, varnish has parameter to limit the size of HTTP request : http_req_size.
But it doesn't specify on which HTTP verb this applies. In theory, the API should accept body only on POST/PUT/PATCH request so it should do the job.
Another solution could be to modify sub vlc_recv in puppet-swh-site/site-modules/profile to cache POST requests and throw error on body size limit as described here.