Page MenuHomeSoftware Heritage

Can't create cooking requests in swh-web
Closed, ResolvedPublic

Description

When on any repository page, eg. https://archive.softwareheritage.org/browse/origin/https://github.com/SoftwareHeritage/swh-core/directory/ , clicking Actions>Downloads>as tarball (or as git) sends a GET to https://archive.softwareheritage.org/api/1/vault/directory/30103e19ba4b9f1c16640d63cab69a3dcd0e8e9f/ , which returns a 500 error and the body is: {"exception":"RemoteException","reason":"404 not found"}

Sentry report: https://sentry.softwareheritage.org/share/issue/e5a8822e0f1f40ca984a3783800369e4/ (shows a GET http://vangogh.euwest.azure.internal.softwareheritage.org:5005/progress/directory/30103e19ba4b9f1c16640d63cab69a3dcd0e8e9f 404ing)

Event Timeline

vlorentz triaged this task as High priority.May 1 2020, 5:51 PM
vlorentz created this task.
vlorentz updated the task description. (Show Details)
vlorentz updated the task description. (Show Details)

Ok I see what is wrong here. Based on the obtained traceback, this is due to changes in how 404 errors are handled server side:

Traceback (most recent call last):
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/apidoc.py", line 358, in documented_view
    response = f(request, **kwargs)
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/views/vault.py", line 100, in api_vault_cook_directory
    res = _dispatch_cook_progress(request, "directory", obj_id)
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/views/vault.py", line 28, in _dispatch_cook_progress
    request=request,
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/views/utils.py", line 67, in api_lookup
    res = lookup_fn(*args)
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/common/service.py", line 1112, in vault_progress
    raise e
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/common/service.py", line 1109, in vault_progress
    return vault.progress(obj_type, obj_id)
  File "/home/antoine/swh/swh-environment/swh-vault/swh/vault/api/client.py", line 29, in progress
    return self.get("progress/{}/{}".format(obj_type, hex_id))
  File "/home/antoine/swh/swh-environment/swh-core/swh/core/api/__init__.py", line 294, in get
    return self._decode_response(response)
  File "/home/antoine/swh/swh-environment/swh-core/swh/core/api/__init__.py", line 352, in _decode_response
    self.raise_for_status(response)
  File "/home/antoine/swh/swh-environment/swh-core/swh/core/api/__init__.py", line 308, in raise_for_status
    raise RemoteException(payload="404 not found", response=response)
swh.core.api.RemoteException: 404 not found

Previously a null response was returned for such cases.

Now a RemoteException is raised stating that a 404 appeared.

Default swh-web behavior for the API is to raise a 404 when a lookup function returns None.

Unfortunately, we missed that issue in the webapp tests as we are forced to mock the vault.

I think the simple way to fix this is to catch the RemoteException for 404 in the api_lookup function to restore previous behavior.
We should also add a new docker test checking that sending a vault GET request on a not yet cooked object efectively returns a 404.

I think the simple way to fix this is to catch the RemoteException for 404 in the api_lookup function to restore previous behavior.

Catching the exception in vault endpoint seems a better choice after more thinking, we could let unrelated errors go through otherwise.

indeed, I forgot to adapt the vault after a refactoring of swh-core. I'll fix it

indeed, I forgot to adapt the vault after a refactoring of swh-core. I'll fix it

I'm already working on the fix, diff incoming.