Page MenuHomeSoftware Heritage
Paste P1469

content loader checksum mismatched
ActivePublic

Authored by ardumont on Oct 4 2022, 10:25 AM.
swh-loader_1 | [2022-10-03 17:03:45,979: INFO/ForkPoolWorker-14] Load origin 'https://sources.debian.net/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip' with type 'content'
swh-loader_1 | [2022-10-03 17:03:45,983: DEBUG/ForkPoolWorker-14] prepare; origin_url=https://sources.debian.net/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip fallback=https://sources.debian.net/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip scheme=https path=/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip
swh-loader_1 | [2022-10-03 17:03:47,000: DEBUG/ForkPoolWorker-14] filename: use_minizip
swh-loader_1 | [2022-10-03 17:03:47,000: DEBUG/ForkPoolWorker-14] filepath: /tmp/tmpbyw8070j/use_minizip
swh-loader_1 | [2022-10-03 17:03:47,002: ERROR/ForkPoolWorker-14] Loading failure, updating to `failed` status
swh-loader_1 | Traceback (most recent call last):
swh-loader_1 | File "/src/swh-loader-core/swh/loader/core/loader.py", line 404, in load
swh-loader_1 | more_data_to_fetch = self.fetch_data()
swh-loader_1 | File "/src/swh-loader-core/swh/loader/core/loader.py", line 730, in fetch_data
swh-loader_1 | file_path, _ = download(url, dest=tmpdir, hashes=self.checksums)
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/tenacity/__init__.py", line 326, in wrapped_f
swh-loader_1 | return self(f, *args, **kw)
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/tenacity/__init__.py", line 406, in __call__
swh-loader_1 | do = self.iter(retry_state=retry_state)
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/tenacity/__init__.py", line 351, in iter
swh-loader_1 | return fut.result()
swh-loader_1 | File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 428, in result
swh-loader_1 | return self.__get_result()
swh-loader_1 | File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
swh-loader_1 | raise self._exception
swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/tenacity/__init__.py", line 409, in __call__
swh-loader_1 | result = fn(*args, **kwargs)
swh-loader_1 | File "/src/swh-loader-core/swh/loader/package/utils.py", line 150, in download
swh-loader_1 | % (url, expected_digest, actual_digest)
swh-loader_1 | ValueError: Failure when fetching https://sources.debian.org/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip. Checksum mismatched: 99ad3828ae35865989a33a8af3c165901889ee9fbf0022434384f1d069d5d70d != 0cedee9714f4295077ee258034dc16b95fd8ecc987e0a75261e5b59054db021a
swh-loader_1 | [2022-10-03 17:03:47,023: DEBUG/ForkPoolWorker-14] cleanup

Event Timeline

Either the manifest referenced the wrong integrity for that urls (or someone updated in
place the urls...):

$ cat /var/tmp/*.json | jq . | grep -C3 "use_minizip"
      "outputHashMode": "flat",
      "type": "url",
      "urls": [
        "https://sources.debian.net/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip"
      ],
      "integrity": "sha256-ma04KK41hlmJozqK88FlkBiJ7p+/ACJDQ4Tx0GnV1w0=",
      "inferredFetcher": "fetchpatch"
--
      "outputHashMode": "flat",
      "type": "url",
      "urls": [
        "https://sources.debian.net/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip"
      ],
      "integrity": "sha256-ma04KK41hlmJozqK88FlkBiJ7p+/ACJDQ4Tx0GnV1w0=",
      "inferredFetcher": "fetchpatch"
$ ipython
...
In [7]: from typing import Dict

In [8]: chksum_algo, chksum_b64 = integrity.split("-")
   ...: checksums: Dict[str, str] = {
   ...:     chksum_algo: base64.decodebytes(chksum_b64.encode()).hex()
   ...: }

In [9]: checksums
Out[9]: {'sha256': '99ad3828ae35865989a33a8af3c165901889ee9fbf0022434384f1d069d5d70d'}

In [10]: exit
$ sha256sum use_minizip
0cedee9714f4295077ee258034dc16b95fd8ecc987e0a75261e5b59054db021a  use_minizip

# cli tools agrees with the python code ^, the integrity filed referenced in the
# manifest is wrong.

# no information on that file from the server (regarding integrity)

$ http head https://sources.debian.net/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip
HTTP/1.1 301 Moved Permanently
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
Date: Tue, 04 Oct 2022 08:30:18 GMT
Keep-Alive: timeout=5, max=100
Location: https://sources.debian.org/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip
Permissions-Policy: interest-cohort=()
Referrer-Policy: no-referrer
Server: Apache
Strict-Transport-Security: max-age=15552000
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
X-Xss-Protection: 1

$ http --follow head https://sources.debian.net/data/main/g/gpsbabel/1.5.3-2/debian/patches/use_minizip
HTTP/1.1 200 OK
Accept-Ranges: bytes
Connection: Upgrade, Keep-Alive
Content-Length: 347
Date: Tue, 04 Oct 2022 08:31:00 GMT
ETag: "15b-5534056fd3380"
Keep-Alive: timeout=5, max=100
Last-Modified: Sat, 01 Jul 2017 12:04:46 GMT
Permissions-Policy: interest-cohort=()
Referrer-Policy: no-referrer
Server: Apache
Strict-Transport-Security: max-age=15552000
Upgrade: h2,h2c
X-Clacks-Overhead: GNU Terry Pratchett
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
X-Xss-Protection: 1

As this one is a "flat" output hash mode, it's actually a real mismatch computation.
This renders nothing to try and use nix-store --dump as [1] does.

[1] P1473