Page MenuHomeSoftware Heritage

deposit: Failure in deposit check
Closed, MigratedEdits Locked

Description

Behavior changed introduced in D3124 (latest version >= v0.0.83).
I think the configuration stanza is what needs to be changed.

Here is the current error:

May 07 14:16:18 worker07 python3[181065]: [2020-05-07 14:16:18,583: INFO/MainProcess] Received task: swh.deposit.loader.tasks.ChecksDepositTsk[f7c8d8f7-b02d-498e-85e4-27ed7dd73827]
May 07 14:16:18 worker07 python3[181073]: [2020-05-07 14:16:18,596: INFO/ForkPoolWorker-1] Loading config file /etc/softwareheritage/global.ini
May 07 14:16:18 worker07 python3[181073]: [2020-05-07 14:16:18,596: INFO/ForkPoolWorker-1] Loading config file /etc/softwareheritage/checker_deposit.yml
May 07 14:16:18 worker07 python3[181073]: [2020-05-07 14:16:18,609: ERROR/ForkPoolWorker-1] Failure during check on '/swh/600/check/'
                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
                                              (self._dns_host, self.port), self.timeout, **extra_kw)
                                            File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 80, in create_connection
                                              raise err
                                            File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 70, in create_connection
                                              sock.connect(sa)
                                          ConnectionRefusedError: [Errno 111] Connection refused

                                          During handling of the above exception, another exception occurred:

                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
                                              chunked=chunked)
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 354, in _make_request
                                              conn.request(method, url, **httplib_request_kw)
                                            File "/usr/lib/python3.7/http/client.py", line 1244, in request
                                              self._send_request(method, url, body, headers, encode_chunked)
                                            File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
                                              self.endheaders(body, encode_chunked=encode_chunked)
                                            File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
                                              self._send_output(message_body, encode_chunked=encode_chunked)
                                            File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
                                              self.send(msg)
                                            File "/usr/lib/python3.7/http/client.py", line 966, in send
                                              self.connect()
                                            File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 181, in connect
                                              conn = self._new_conn()
                                            File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
                                              self, "Failed to establish a new connection: %s" % e)
                                          urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fa239b11550>: Failed to establish a new connection: [Errno 111] Connection refused

                                          During handling of the above exception, another exception occurred:

                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send
                                              timeout=timeout
                                            File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
                                              _stacktrace=sys.exc_info()[2])
                                            File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
                                              raise MaxRetryError(_pool, url, error or ResponseError(cause))
                                          urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=5006): Max retries exceeded with url: /1/private/swh/600/check/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa239b11550>: Failed to establish a new connection: [Errno 111] Connection refused'))

                                          During handling of the above exception, another exception occurred:

                                          Traceback (most recent call last):
                                            File "/usr/lib/python3/dist-packages/swh/deposit/loader/checker.py", line 44, in check
                                              r = self.client.check(deposit_check_url)
                                            File "/usr/lib/python3/dist-packages/swh/deposit/client.py", line 210, in check
                                              r = self.do("get", check_url)
                                            File "/usr/lib/python3/dist-packages/swh/deposit/client.py", line 114, in do
                                              return method_fn(full_url, *args, **kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/api.py", line 75, in get
                                              return request('get', url, params=params, **kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
                                              return session.request(method=method, url=url, **kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
                                              resp = self.send(prep, **send_kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
                                              r = adapter.send(request, **kwargs)
                                            File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
                                              raise ConnectionError(e, request=request)
                                          requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5006): Max retries exceeded with url: /1/private/swh/600/check/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa239b11550>: Failed to establish a new connection: [Errno 111] Connection refused'))
May 07 14:16:18 worker07 python3[181073]: [2020-05-07 14:16:18,650: INFO/ForkPoolWorker-1] Task swh.deposit.loader.tasks.ChecksDepositTsk[f7c8d8f7-b02d-498e-85e4-27ed7dd73827] succeeded in 0.038403804996050894s: {'status': 'failed'}

Event Timeline

ardumont triaged this task as Unbreak Now! priority.
ardumont updated the task description. (Show Details)
ardumont added a project: SWORD deposit.

I think the configuration stanza is what needs to be changed.

Yes, it is.

Introducing back the SWHConfig mixin (nightmare) in the DepositChecker. That
now bypasses the default configuration from the collaborator
PrivateApiDepositClient (whose base class already had some configuration
stanza).

It also introduced a configuration structure change.

I missed both those subtle change during reviews.

After that pirouette, it's actually clearer now.

moranegg moved this task from Deployed to Archived on the SWORD deposit board.