The PyPI loader in production fails to parse PKG-INFO files with non UTF-8 encoding, see related sentry issue.

Nevertheless, the issue do not occur in docker environment as we are using the latest upstream version of pkginfo fetched from PyPI.

If we look at pkginfo development history, we can see the issue has been fixed in revision 155 and since release 1.5.0.

Unfortunately, debian packages pkginfo 1.4.2 in buster and bullseye so numerous PKG-INFO files fail to be parsed.

We should upgrade python3-pkginfo debian package to fix those parsing issues.

Other solution would be to use codecs.register_error to workaround the issue but this is really hackish.