Page MenuHomeSoftware Heritage

Sending a POST to the /save/git/url API may return a 403
Closed, MigratedEdits Locked

Description

When using the public API to send a save code now request, one may encounter a Forbidden response is the Referer header is not present.

curl -D-  -X POST  'https://archive.softwareheritage.org/save/git/url/https://git.joeyh.name/git/ikiwiki.git//' -H 'Accept: application/json' --data ''
HTTP/1.1 403 Forbidden
Date: Fri, 15 Nov 2019 12:34:48 GMT
Server: gunicorn/19.9.0
Content-Type: application/json
Vary: Accept
Allow: OPTIONS, POST
X-Frame-Options: SAMEORIGIN
Content-Length: 63
Via: 1.1 archive.softwareheritage.org
X-Varnish: 6599915
Age: 0
Via: 1.1 varnish (Varnish/6.1)
Strict-Transport-Security: max-age=15768000;
Connection: keep-alive

{"detail":"CSRF Failed: Referer checking failed - no Referer."}

In a web browser, the response is displayed as

The "save code now" request has been rejected because the provided origin url is blacklisted.

which is wrong:

To reproduce with Firefox, set the network.http.sendRefererHeader config to 1 then submit a save code now request via https://archive.softwareheritage.org/save/

Event Timeline

douardda triaged this task as Normal priority.Nov 15 2019, 1:39 PM
douardda created this task.

This is an expected behaviour as https://archive.softwareheritage.org/save/ links to the HTML form to submit a save request, this is not an api endpoint.

To submit a save code now request through the API, you have to do:

$ curl -X POST https://archive.softwareheritage.org/api/1/origin/save/git/url/https://git.joeyh.name/git/ikiwiki.git/

Nevertheless, the displayed message is of course wrong and it should have said that the request is under pending state.
I need to check why this happened.

Otherwise I added https://git.joeyh.name/ as authorized url prefixes for save code now requests so next submitted requests will be automatically accepted.

This is an expected behaviour as https://archive.softwareheritage.org/save/ links to the HTML form to submit a save request, this is not an api endpoint.

Ok, there is effectively an issue with the Save code now form when the Referer HTTP header is not sent.

Actually, this is Django normal behavior regarding CSRF validation, quoting its documentation:

In addition, for HTTPS requests, strict referer checking is done by CsrfViewMiddleware. 
This means that even if a subdomain can set or modify cookies on your domain, it can’t 
force a user to post to your application since that request won’t come from your own exact domain.

This also addresses a man-in-the-middle attack that’s possible under HTTPS when using 
a session independent secret, due to the fact that HTTP Set-Cookie headers are 
(unfortunately) accepted by clients even when they are talking to a site under HTTPS. 
(Referer checking is not done for HTTP requests because the presence of the Referer 
header isn’t reliable enough under HTTP.)

The Referer checking stuff is built-in to Django's CSRF protection, and I don't think we should disable it.

https://docs.djangoproject.com/en/1.11/ref/csrf/#how-it-works point 4.

I can reproduce the issue when setting http://kb.mozillazine.org/Network.http.sendRefererHeader to 1. I expect the referer is not transmitted because the form submission is intercepted by javascript instead of being done via a plain form POST.