Symmetrically to other loader reports, here is one on the current redundant failures we have on our git origins.
The period covered is from August to October 2018:
$ cat git-output-august-october-2018.txt | python3 -m group_by_exception --loader-type git --no-aggregate | jq . { "total": 1167689, "errors": { "FileNotFoundError(2, 'No such file or directory')": 648242, // <- worker03 had a missing mount point, had the issue with mercurial loader "dulwich.errors.GitProtocolError: unexpected http resp 401 for": 403090, "psycopg2.extensions.QueryCanceledError: canceling statement due to statement timeout": 36628, "502 Bad Gateway": 34300, "gaierror(-3, 'Temporary failure in name resolution')": 17493, "consumer: Cannot connect to amqp": 6649, "pgbouncer cannot connect to server": 4811, "StorageAPIError(ConnectionError(MaxRetryError('None: Max retries exceeded with url: ": 3307, "Worker exited prematurely: signal 9": 2749, "MemoryError": 2164, "ValueError: invalid literal for int() with base 16: b''": 1898, "swh.objstorage.exc.ObjStorageAPIError: An unexpected error occurred in the api backend:": 1309, "OSError: Pack file too big for repository": 978, "TypeError: snapshot_add() got an unexpected keyword argument 'back_compat'": 890, "ConnectionResetError: [Errno 104] Connection reset by peer": 597, "socket.gaierror: [Errno -3] Temporary failure in name resolution": 563, "http.client.IncompleteRead:": 509, "dulwich.errors.NotGitRepository": 231, "BrokenPipeError: [Errno 32] Broken pipe": 208, "dulwich.errors.GitProtocolError: unexpected http resp 503": 168, "504 Gateway Time-out": 127, "dulwich.errors.ObjectFormatException: invalid literal for int() with base 10:": 115, "StorageDBError(OperationalError('could not connect to server: Connection refused": 113, "psycopg2.IntegrityError: duplicate key value violates unique constraint ": 102, "ConnectionResetError(104, 'Connection reset by peer')": 89, "psycopg2.InternalError: current transaction is aborted, commands ignored until end of transaction block": 53, "psycopg2.extensions.TransactionRollbackError: deadlock detected": 39, "psycopg2.IntegrityError: null value in column ": 38, "SSL connection has been closed unexpectedly": 29, "socket.gaierror: [Errno -5] No address associated with hostname": 25, "psycopg2.DatabaseError: server conn crashed?": 22, "ValueError: invalid literal for int() with base 16:": 20, "OSError(ConnectionRefusedError(111, 'Connection refused')": 15, "ValueError: year is out of range": 14, "KeyError": 14, "OSError(timeout('timed out',),)": 12, "OSError(113, 'No route to host')": 12, "dulwich.errors.GitProtocolError: unexpected http resp 500": 11, "TimeoutError: [Errno 110] Connection timed out": 8, "Worker exited prematurely: signal 15 (SIGTERM).',)": 6, "socket.gaierror: [Errno -2] Name or service not known": 5, "dulwich.errors.GitProtocolError: unexpected http resp 502 for": 4, "dulwich.errors.HangupException: The remote server unexpectedly closed the connection.": 4, "TimeoutError(110, 'Connection timed out')": 4, "OSError: [Errno 113] No route to host": 3, "ValueError: Wrong content type `text/html` for API response": 3, "psycopg2.OperationalError: terminating connection due to administrator command": 2, "OverflowError: timestamp out of range for platform time_t": 2, "amqp.exceptions.NotFound: Basic.publish: (404) NOT_FOUND": 2, "CONNECTION_FORCED - broker forced connection closure with reason": 2, "OSError(OSError(113, 'No route to host'),)": 2, "Timed out waiting for UP message from": 1, "psycopg2.OperationalError: index row size": 1, "dulwich.errors.ObjectFormatException: Unknown field": 1, "dulwich.errors.GitProtocolError: unexpected http resp 429": 1, "gaierror(-2, 'Name or service not known')": 1, "OSError(5, 'Input/output error')": 1, "msgpack.exceptions.UnpackValueError: Unpack failed: error = 0": 1, "ValueError: not enough values to unpack": 1 } }
As usual, some are issues, some are not (e.g 401)
Associated to the error is the number of occurrences.
That will help to determine the next issue to fix.
This is a meta-task to open and track the associated issue if it is deemed one.
Related T1246#23774
Related T1159#23772