Can't create cooking requests in swh-web
Closed, MigratedEdits Locked
Actions

Assigned To

Authored By

	vlorentz
	May 1 2020, 5:51 PM

Description

When on any repository page, eg. https://archive.softwareheritage.org/browse/origin/https://github.com/SoftwareHeritage/swh-core/directory/ , clicking Actions>Downloads>as tarball (or as git) sends a GET to https://archive.softwareheritage.org/api/1/vault/directory/30103e19ba4b9f1c16640d63cab69a3dcd0e8e9f/ , which returns a 500 error and the body is: {"exception":"RemoteException","reason":"404 not found"}

Sentry report: https://sentry.softwareheritage.org/share/issue/e5a8822e0f1f40ca984a3783800369e4/ (shows a GET http://vangogh.euwest.azure.internal.softwareheritage.org:5005/progress/directory/30103e19ba4b9f1c16640d63cab69a3dcd0e8e9f 404ing)

Revisions and Commits

rDWAPPS Web applications
		D3125	rDWAPPSaffd506bc632 api/vault: Fix 404 handling and improve error messages
rDVAU Software Heritage Vault
	Closed		D3127 Raise NotFoundExc within our RPC framework instead of returning 404.

Related Objects

Mentioned In: D3125: api/vault: Fix 404 handling and improve error messages

Event Timeline

vlorentz triaged this task as High priority.May 1 2020, 5:51 PM

vlorentz created this task.

vlorentz updated the task description. (Show Details)

Ok I see what is wrong here. Based on the obtained traceback, this is due to changes in how 404 errors are handled server side:

Traceback (most recent call last):
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/apidoc.py", line 358, in documented_view
    response = f(request, **kwargs)
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/views/vault.py", line 100, in api_vault_cook_directory
    res = _dispatch_cook_progress(request, "directory", obj_id)
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/views/vault.py", line 28, in _dispatch_cook_progress
    request=request,
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/api/views/utils.py", line 67, in api_lookup
    res = lookup_fn(*args)
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/common/service.py", line 1112, in vault_progress
    raise e
  File "/home/antoine/swh/swh-environment/swh-web/swh/web/common/service.py", line 1109, in vault_progress
    return vault.progress(obj_type, obj_id)
  File "/home/antoine/swh/swh-environment/swh-vault/swh/vault/api/client.py", line 29, in progress
    return self.get("progress/{}/{}".format(obj_type, hex_id))
  File "/home/antoine/swh/swh-environment/swh-core/swh/core/api/__init__.py", line 294, in get
    return self._decode_response(response)
  File "/home/antoine/swh/swh-environment/swh-core/swh/core/api/__init__.py", line 352, in _decode_response
    self.raise_for_status(response)
  File "/home/antoine/swh/swh-environment/swh-core/swh/core/api/__init__.py", line 308, in raise_for_status
    raise RemoteException(payload="404 not found", response=response)
swh.core.api.RemoteException: 404 not found

Previously a null response was returned for such cases.

Now a RemoteException is raised stating that a 404 appeared.

Default swh-web behavior for the API is to raise a 404 when a lookup function returns None.

Unfortunately, we missed that issue in the webapp tests as we are forced to mock the vault.

I think the simple way to fix this is to catch the RemoteException for 404 in the api_lookup function to restore previous behavior.
We should also add a new docker test checking that sending a vault GET request on a not yet cooked object efectively returns a 404.

I think the simple way to fix this is to catch the RemoteException for 404 in the api_lookup function to restore previous behavior.

Catching the exception in vault endpoint seems a better choice after more thinking, we could let unrelated errors go through otherwise.

indeed, I forgot to adapt the vault after a refactoring of swh-core. I'll fix it

indeed, I forgot to adapt the vault after a refactoring of swh-core. I'll fix it

I'm already working on the fix, diff incoming.

anlambert added a revision: D3125: api/vault: Fix 404 handling and improve error messages.May 5 2020, 4:12 PM

anlambert mentioned this in D3125: api/vault: Fix 404 handling and improve error messages.May 5 2020, 5:08 PM

anlambert closed this task as Resolved by committing rDWAPPSaffd506bc632: api/vault: Fix 404 handling and improve error messages.May 5 2020, 7:20 PM

anlambert claimed this task.

anlambert added a commit: rDWAPPSaffd506bc632: api/vault: Fix 404 handling and improve error messages.

vlorentz added a revision: D3127: Raise NotFoundExc within our RPC framework instead of returning 404..May 11 2020, 11:33 AM

tenma moved this task from Backlog to Done on the Vault board.Oct 16 2020, 11:33 AM

This task has been migrated to GitLab.

Can't create cooking requests in swh-webClosed, MigratedEdits LockedActions

Description

Revisions and Commits

Related Objects

Event Timeline

Can't create cooking requests in swh-web
Closed, MigratedEdits Locked
Actions