Page MenuHomeSoftware Heritage
Create Task
Maniphest T2401

deposit.client: Improve error message instead of xml parsing failure
Closed, MigratedEdits Locked
Actions

Description

Currently, our deposit backends (not the api, the backend below) is down due to
maintenance [1]

This results in unrelated parsing failure issues in the deposit clients:

Traceback (most recent call last):
  File "/home/user/swh/python3env/bin/swh", line 8, in <module>
    sys.exit(main())
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/core/cli/__init__.py", line 122, in main
    return swh(auto_envvar_prefix="SWH")
  File "/home/user/swh/python3env/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/swh/python3env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/swh/python3env/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/deposit/cli/client.py", line 440, in upload
    r = deposit_create(config, logger)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/deposit/cli/client.py", line 274, in deposit_create
    return client.deposit_create(**_subdict(config, keys))
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/deposit/client.py", line 582, in deposit_create
    collection, archive, metadata, in_progress, slug
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/deposit/client.py", line 304, in execute
    error = self.parse_result_error(r.text)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/deposit/client.py", line 267, in parse_result_error
    xml_content, keys=["summary", "detail", "sword:verboseDescription"]
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/deposit/client.py", line 61, in _parse_with_filter
    data = _parse(stream, encoding=encoding)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/swh/deposit/client.py", line 39, in _parse
    data = xmltodict.parse(stream, encoding=encoding, process_namespaces=False)
  File "/home/user/swh/python3env/lib/python3.7/site-packages/xmltodict.py", line 327, in parse
    parser.Parse(xml_input, True)
xml.parsers.expat.ExpatError: syntax error: line 1, column 49

This needs to be improved.

[1] https://status.softwareheritage.org/

Revisions and Commits

rDDEP Push deposit
ClosedD3139 Make deposit client deal properly with maintenance issues

Related Objects

Event Timeline

ardumont triaged this task as Normal priority.May 11 2020, 10:57 AM
ardumont created this task.
ardumont mentioned this in rDDEPc64ee54311f7: Add deposit exception handler to improve default error display.May 11 2020, 3:28 PM
ardumont mentioned this in rDDEP6a2759e50c76: Align 503 exceptions output format with existing errors.
ardumont mentioned this in P666 swh-deposit: much better error message when backend maintenance detected.May 11 2020, 5:06 PM
ardumont added a revision: D3139: Make deposit client deal properly with maintenance issues.May 11 2020, 6:04 PM
ardumont added a comment.May 11 2020, 6:08 PM

Fixed the server side api error displayed (deployed):

$ curl -i -u user:"<pass>" https://deposit.softwareheritage.org/1/servicedocument/
HTTP/1.1 503 Service Unavailable
Date: Mon, 11 May 2020 16:05:57 GMT
Server: gunicorn/19.9.0
Content-Type: application/xml
Vary: Accept
Allow: GET, POST, PUT, DELETE, HEAD, OPTIONS
X-Frame-Options: SAMEORIGIN
Content-Length: 311
Via: 1.1 deposit.softwareheritage.org
Access-Control-Allow-Origin: *
X-Varnish: 132860
Age: 0
Via: 1.1 varnish (Varnish/6.1)
Strict-Transport-Security: max-age=15768000;
Connection: keep-alive

<?xml version="1.0" encoding="utf-8"?>
<sword:error xmlns="http://www.w3.org/2005/Atom"
       xmlns:sword="http://purl.org/net/sword/">
    <summary>Database backend maintenance</summary>
    <sword:verboseDescription>Service temporarily unavailable, try again later.</sword:verboseDescription>
</sword:error>

The deposit api client has been fixed to properly display the message when this
occurs:

$ swh deposit status --username user --password "<pass>" --deposit-id 589 --format json
ERROR:swh.deposit.cli.client:Database backend maintenance: Service temporarily unavailable, try again later.

No more noisy parsing issues which has nothing to do with the user input.

ardumont closed this task as Resolved.May 11 2020, 6:09 PM
ardumont claimed this task.
ardumont moved this task from Backlog to Archived on the SWORD deposit board.Nov 3 2020, 4:06 PM
gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 4:30 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal