Page MenuHomeSoftware Heritage

API endpoint /vault/directory/<dirhash>/ should not be cached by varnish
Closed, MigratedEdits Locked

Description

At very least, because this looks weird:

$ http https://archive.softwareheritage.org/api/1/vault/directory/f557a108b5878c7d7a46e94f538d6bb65379e62e/
HTTP/1.1 404 Not Found
Age: 0
Allow: POST, GET, OPTIONS
Connection: keep-alive
Content-Length: 123
Content-Type: application/json
Date: Wed, 25 Nov 2020 08:55:53 GMT
Server: gunicorn/19.9.0
Strict-Transport-Security: max-age=15768000;
Vary: Accept,Cookie
Via: 1.1 archive.softwareheritage.org
Via: 1.1 varnish (Varnish/6.1)
X-Frame-Options: SAMEORIGIN
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 58
X-RateLimit-Reset: 1606294554
X-Varnish: 213079

{
    "exception": "NotFoundExc",
    "reason": "Cooking of directory 'f557a108b5878c7d7a46e94f538d6bb65379e62e' was never requested."
}

$ http POST https://archive.softwareheritage.org/api/1/vault/directory/f557a108b5878c7d7a46e94f538d6bb65379e62e/
HTTP/1.1 200 OK
[...]

$ http https://archive.softwareheritage.org/api/1/vault/directory/f557a108b5878c7d7a46e94f538d6bb65379e62e/
HTTP/1.1 404 Not Found
Age: 99
Allow: POST, GET, OPTIONS
Connection: keep-alive
Content-Length: 123
Content-Type: application/json
Date: Wed, 25 Nov 2020 08:55:53 GMT
Server: gunicorn/19.9.0
Strict-Transport-Security: max-age=15768000;
Vary: Accept,Cookie
Via: 1.1 archive.softwareheritage.org
Via: 1.1 varnish (Varnish/6.1)
X-Frame-Options: SAMEORIGIN
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 58
X-RateLimit-Reset: 1606294554
X-Varnish: 1515525 213080

{
    "exception": "NotFoundExc",
    "reason": "Cooking of directory 'f557a108b5878c7d7a46e94f538d6bb65379e62e' was never requested."
}

Event Timeline

douardda created this task.
douardda added a subscriber: olasd.

This may be better suited to a fix in the web API, via proper cache config headers, as @olasd mentioned on IRC (probably via https://docs.djangoproject.com/en/3.1/topics/cache/#downstream-caches )

This may be better suited to a fix in the web API, via proper cache config headers, as @olasd mentioned on IRC (probably via https://docs.djangoproject.com/en/3.1/topics/cache/#downstream-caches )

This is already the case: https://forge.softwareheritage.org/source/swh-web/browse/master/swh/web/api/views/vault.py$57

There is clearly a regression at the Varnish level as vault cooking progress was correctly reported in the Web UI before.

I am wondering if it could be a side effect of the recent setting of CORS headers for the api paths.

There is clearly a regression at the Varnish level as vault cooking progress was correctly reported in the Web UI before.

I am wondering if it could be a side effect of the recent setting of CORS headers for the api paths.

Turned out the issue was at the Django level due to a recent refactoring and missing tests ... Fixed in D4595.