Change Details

Multiple types of issues are currently reported in our sentry instance since the loading started. Opened here so they are publicly shareable [1] (not investigated). I've added [3] which is all the last log of the failing worker which should give the origin in failure plus the actual encoutered issue. - [[ https://sentry.softwareheritage.org/share/issue/57a8778ba1bc4cf4b9ff4e489f1fa176/ | Problem during unpacking ...mssql-2.1.0.tbz. Reason: Unknown archive format '.../mssql-2.1.0.tbz'... ]] - [[ https://sentry.softwareheritage.org/share/issue/57a8778ba1bc4cf4b9ff4e489f1fa176/ | Problem during unpacking /tmp/tmp9qvp688z/clustershell-1.3.2-1.fc13.src.rpm. Reason: Unknown archive format '/tmp/tmp9qvp688z/clustershell-1.3.2-1.fc13.src.rpm' ]] - [[ https://sentry.softwareheritage.org/share/issue/c3e95bf9fea94038805cd5160d7da3a8/ | Fail to query 'http://www-verimag.imag.fr/DIST-TOOLS/SYNCHRONE/pool/rdbg.1.70.tgz'. Reason: 404 ]] [2] - [[ https://sentry.softwareheritage.org/share/issue/c3e95bf9fea94038805cd5160d7da3a8/ | Fail to query 'http://www.lexifi.com/sedlex/sedlex-1.99.1.tar.gz'. Reason: 404 ]] - [[ https://sentry.softwareheritage.org/share/issue/beb647ddb82c43a9930ab16ff5278079/ | HTTPSConnectionPool(host='gitlab.inria.fr', port=443): Max retries exceeded with url: /fpottier/inferno/-/archive/20201104/ ]] - [[ https://sentry.softwareheritage.org/share/issue/008fbab7d228449ea1659a45de48d279/ | HTTPSConnectionPool(host='opam.ocaml.org', port=443): Max retries exceeded with url: /cache/md5/f6/f69ee4f693330 ]] From afar, the 404, we cannot do much about it. We have at least 3 unsupported archive formats "rpm", "tbz" and "tgz". Fixing those sound like the most important. Plus it's beneficial for other package loaders (e.g archive, cran, pypi, nixguix, ...). The connection error ones might be worked around adding some retry decorators like those existing in lister. [1] kibana is not opened so my dashboard opening was not that helpful... [2] full extract of all events "so far" in F4628907 (contains more than just opam tasks). [3] F4628948

Multiple types of issues are currently reported in our sentry instance since the loading started. Opened here so they are publicly shareable [1] (not investigated). I've added [3] which is all the last log of the failing worker which should give the origin in failure plus the actual encoutered issue. - [[ https://sentry.softwareheritage.org/share/issue/57a8778ba1bc4cf4b9ff4e489f1fa176/ | Problem during unpacking ...mssql-2.1.0.tbz. Reason: Unknown archive format '.../mssql-2.1.0.tbz'... ]] - [[ https://sentry.softwareheritage.org/share/issue/57a8778ba1bc4cf4b9ff4e489f1fa176/ | Problem during unpacking /tmp/tmp9qvp688z/clustershell-1.3.2-1.fc13.src.rpm. Reason: Unknown archive format '/tmp/tmp9qvp688z/clustershell-1.3.2-1.fc13.src.rpm' ]] - [[ https://sentry.softwareheritage.org/share/issue/c3e95bf9fea94038805cd5160d7da3a8/ | Fail to query 'http://www-verimag.imag.fr/DIST-TOOLS/SYNCHRONE/pool/rdbg.1.70.tgz'. Reason: 404 ]] [2] - [[ https://sentry.softwareheritage.org/share/issue/c3e95bf9fea94038805cd5160d7da3a8/ | Fail to query 'http://www.lexifi.com/sedlex/sedlex-1.99.1.tar.gz'. Reason: 404 ]] - [[ https://sentry.softwareheritage.org/share/issue/beb647ddb82c43a9930ab16ff5278079/ | HTTPSConnectionPool(host='gitlab.inria.fr', port=443): Max retries exceeded with url: /fpottier/inferno/-/archive/20201104/ ]] - [[ https://sentry.softwareheritage.org/share/issue/008fbab7d228449ea1659a45de48d279/ | HTTPSConnectionPool(host='opam.ocaml.org', port=443): Max retries exceeded with url: /cache/md5/f6/f69ee4f693330 ]] From afar, the 404, we cannot do much about it (P1117#7495 for the origins in question). We have at least 3 unsupported archive formats "rpm", "tbz" and "tgz". Fixing those sound like the most important. Plus it's beneficial for other package loaders (e.g archive, cran, pypi, nixguix, ...). The connection error ones might be worked around adding some retry decorators like those existing in lister. [1] kibana is not opened so my dashboard opening was not that helpful... [2] full extract of all events "so far" in F4628907 (contains more than just opam tasks). [3] F4628948