The main issue that prevents us from archiving these objects today is that our object storage still uses a plain sha1 as primary key (hence the current unicity constraint on the sha1 field of the content table in our primary graph storage).
- Queries
- All Stories
- Search
- Advanced Search
- Transactions
- Transaction Logs
Advanced Search
Jan 13 2022
Jan 10 2022
Dec 13 2021
Dec 8 2021
Updated task name and description to reflect the findings from @anlambert
Dec 7 2021
It is possible that more key cryptographic software will include these files.
Thanks a lot @anlambert for looking into this.
It is possible that more key cryptographic software will include these files.
We need a strategy to handle this situation, may you add this example to the SWHID v2 task?
Loading the repository in docker environment gives me the following traceback:
swh-loader_1 | [2021-12-07 12:08:06,876: INFO/MainProcess] Task swh.loader.git.tasks.UpdateGitRepository[a1aa28c0-1cb0-4e2a-8ae2-720ba6ca439e] received swh-loader_1 | [2021-12-07 12:08:06,877: INFO/MainProcess] loader@b11bfd448510 ready. swh-loader_1 | [2021-12-07 12:08:06,957: DEBUG/ForkPoolWorker-1] Loading config file /loader.yml swh-loader_1 | [2021-12-07 12:08:09,904: INFO/ForkPoolWorker-1] Load origin 'https://gitlab.com/sequoia-pgp/sequoia' with type 'git' swh-loader_1 | [2021-12-07 12:08:09,908: DEBUG/ForkPoolWorker-1] Transport url to communicate with server: https://gitlab.com/sequoia-pgp/sequoia swh-loader_1 | [2021-12-07 12:08:09,909: DEBUG/ForkPoolWorker-1] Client Urllib3HttpGitClient('https://gitlab.com/sequoia-pgp/sequoia/', dumb=None) to fetch pack at /sequoia-pgp/sequoia swh-loader_1 | [2021-12-07 12:08:10,422: DEBUG/ForkPoolWorker-1] local_heads_count=0 swh-loader_1 | [2021-12-07 12:08:10,422: DEBUG/ForkPoolWorker-1] remote_heads_count=1821 swh-loader_1 | [2021-12-07 12:08:10,422: DEBUG/ForkPoolWorker-1] wanted_refs_count=1821 swh-loader_1 | [2021-12-07 12:09:17,112: ERROR/ForkPoolWorker-1] Loading failure, updating to `failed` status swh-loader_1 | Traceback (most recent call last): swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/api/client.py", line 29, in raise_for_status swh-loader_1 | super().raise_for_status(response) swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/api/__init__.py", line 344, in raise_for_status swh-loader_1 | raise exception from None swh-loader_1 | swh.core.api.RemoteException: <RemoteException 500 HashCollision: ['sha1', '38762cf7f55934b34d179ae6a4c80cadccbb7f0a', [{'blake2s256': '30e4bd16c3f98e74429d237c19ca9def702e5720cb124cb4b92e74f989aaf116', 'sha1': '38762cf7f55934b34d179ae6a4c80cadccbb7f0a', 'sha1_git': 'b621eeccd5c7edac9b7dcba35a8d5afd075e24f2', 'sha256': 'd4488775d29bdef7993367d541064dbdda50d383f89f0aa13a6ff2e0894ba5ff'}, {'blake2s256': '8f677e3214ca8b2acad91884a1571ef3f12b786501f9a6bedfd6239d82095dd2', 'sha1': '38762cf7f55934b34d179ae6a4c80cadccbb7f0a', 'sha1_git': 'ba9aaa145ccd24ef760cf31c74d8f7ca1a2e47b0', 'sha256': '2bb787a73e37352f92383abe7e2902936d1059ad9f1ba6daaa9c1e58ee6970d0'}]]> swh-loader_1 | swh-loader_1 | During handling of the above exception, another exception occurred: swh-loader_1 | swh-loader_1 | Traceback (most recent call last): swh-loader_1 | File "/src/swh-loader-core/swh/loader/core/loader.py", line 339, in load swh-loader_1 | self.store_data() swh-loader_1 | File "/src/swh-loader-core/swh/loader/core/loader.py", line 458, in store_data swh-loader_1 | self.storage.directory_add([directory]) swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/buffer.py", line 171, in directory_add swh-loader_1 | stats = self.object_add(directories, object_type="directory", keys=["id"]) swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/buffer.py", line 224, in object_add swh-loader_1 | return self.flush() swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/buffer.py", line 286, in flush swh-loader_1 | stats = add_fn(list(batch)) swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/proxies/filter.py", line 58, in content_add swh-loader_1 | [x for x in content if x.sha256 in contents_to_add] swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/api/client.py", line 45, in content_add swh-loader_1 | return self.post("content/add", {"content": content}) swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/api/__init__.py", line 278, in post swh-loader_1 | return self._decode_response(response) swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/core/api/__init__.py", line 354, in _decode_response swh-loader_1 | self.raise_for_status(response) swh-loader_1 | File "/srv/softwareheritage/venv/lib/python3.7/site-packages/swh/storage/api/client.py", line 39, in raise_for_status swh-loader_1 | raise HashCollision(*e.args[0]["args"]) swh-loader_1 | swh.storage.exc.HashCollision: ('sha1', '38762cf7f55934b34d179ae6a4c80cadccbb7f0a', [{'sha256': 'd4488775d29bdef7993367d541064dbdda50d383f89f0aa13a6ff2e0894ba5ff', 'sha1': '38762cf7f55934b34d179ae6a4c80cadccbb7f0a', 'sha1_git': 'b621eeccd5c7edac9b7dcba35a8d5afd075e24f2', 'blake2s256': '30e4bd16c3f98e74429d237c19ca9def702e5720cb124cb4b92e74f989aaf116'}, {'sha256': '2bb787a73e37352f92383abe7e2902936d1059ad9f1ba6daaa9c1e58ee6970d0', 'sha1': '38762cf7f55934b34d179ae6a4c80cadccbb7f0a', 'sha1_git': 'ba9aaa145ccd24ef760cf31c74d8f7ca1a2e47b0', 'blake2s256': '8f677e3214ca8b2acad91884a1571ef3f12b786501f9a6bedfd6239d82095dd2'}])
Dec 3 2021
Nov 9 2021
Moving towards event notifications and stream processing instead of polling sounds worthwhile, before the amount of polling becomes more important than managing the event notification mechanism. For the two systems you've mentioned, I think we're really, really far away from that, but it's still worth considering a way to do event-driven notifications properly, so we don't have to rush it through.
Oct 26 2021
Fixed and deployed, origin https://git.code.sf.net/u/bsomervi/hamlib.git has been successfully submitted and is currently being loaded into the archive, closing this.
This is because I made the wrong assumption that URL paths to git repos for the git.code.sf.net netloc are necessarily prefixed by /p/.
So URLs starting with https://git.code.sf.net/u/ are currently rejected when submitting save requests through the Web UI , see related frontend code.
Oct 20 2021
It has been implemented and deployed to production, closing this.
It has been implemented and deployed to production, closing this.
Oct 19 2021
Oct 15 2021
It's regularly crawled now so closing this.
Oct 11 2021
Issues related to git dumb loading have been handled and the 5 dumb origins that were failing have been resubmitted through save code now and successfully loaded, closing this.
Oct 8 2021
Oct 6 2021
Oct 5 2021
Oct 4 2021
*very old comment not pushed for some reason*
Oct 2 2021
Oct 1 2021
It resolved itself, it's green again.
Sep 29 2021
All save requests have been processed, 127 succeeded and 5 failed due to the following issues:
- https://git.qsomula.top/SUGAR__STRAWBERRIES/SUGAR__AND__STRAWBERRIES.git is an empty repo, that edge case is not properly handled
- 4 git repositories whose URLs are prefixed by https://git.zap.org.au/ reference pack files that are no longer hosted on the server, skipping those missing pack files makes the loadings succeeded
This is the execution result of the script above:
anlambert@carnavalet:~/tmp$ python3 reload_dumb_git_origins.py {'loading_task_id': 400298414, 'origin_url': 'https://git.pengutronix.de/cgit/tools/libsocketcan', 'save_request_date': '2021-09-29T10:07:54.080280+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298415, 'origin_url': 'https://git.stoutner.com/UltraPrivacy.git', 'save_request_date': '2021-09-29T10:07:54.507288+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298416, 'origin_url': 'https://git.stoutner.com/UltraList.git', 'save_request_date': '2021-09-29T10:07:54.795405+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298417, 'origin_url': 'https://git.stoutner.com/PrivacyCell.git', 'save_request_date': '2021-09-29T10:07:54.979121+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298418, 'origin_url': 'https://git.stoutner.com/PrivacyBrowserROMIntegration.git', 'save_request_date': '2021-09-29T10:07:55.205037+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298419, 'origin_url': 'https://git.stoutner.com/PrivacyBrowser.git', 'save_request_date': '2021-09-29T10:07:55.404696+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298420, 'origin_url': 'https://git.raptorengineering.com/git/hqemu', 'save_request_date': '2021-09-29T10:07:55.627926+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298421, 'origin_url': 'https://forge.touhey.org/casio/packaging.git', 'save_request_date': '2021-09-29T10:07:55.919487+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298422, 'origin_url': 'https://forge.touhey.org/casio/libcasio.git', 'save_request_date': '2021-09-29T10:07:56.202117+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298423, 'origin_url': 'https://git.tuxfamily.org/harmonist/harmonist.git', 'save_request_date': '2021-09-29T10:07:56.482737+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298424, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tlibs.git', 'save_request_date': '2021-09-29T10:07:57.516829+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298425, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/miezetools.git', 'save_request_date': '2021-09-29T10:07:57.811821+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298424, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tlibs.git', 'save_request_date': '2021-09-29T10:07:57.516829+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298426, 'origin_url': 'https://depp.brause.cc/nov.el.git', 'save_request_date': '2021-09-29T10:07:58.360459+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298427, 'origin_url': 'https://www.6809.org.uk/git/asm6809.git', 'save_request_date': '2021-09-29T10:07:58.925928+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298428, 'origin_url': 'http://git.ghostscript.com/mupdf.git', 'save_request_date': '2021-09-29T10:07:59.179198+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298429, 'origin_url': 'https://git.joeyh.name/git/oldusenet.git', 'save_request_date': '2021-09-29T10:07:59.423211+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298430, 'origin_url': 'https://git.drobilla.net/ganv.git', 'save_request_date': '2021-09-29T10:07:59.660229+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298431, 'origin_url': 'https://genome-source.gi.ucsc.edu/samtabix.git', 'save_request_date': '2021-09-29T10:07:59.901717+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298432, 'origin_url': 'http://software.legiasoft.com/git/quickcheck.git', 'save_request_date': '2021-09-29T10:08:00.111266+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298433, 'origin_url': 'https://git.progress-linux.org/users/daniel.baumann/debian/packages/lziprecover', 'save_request_date': '2021-09-29T10:08:00.336308+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298434, 'origin_url': 'https://git.progress-linux.org/users/daniel.baumann/debian/packages/plzip', 'save_request_date': '2021-09-29T10:08:00.538700+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298435, 'origin_url': 'https://git.progress-linux.org/users/daniel.baumann/debian/packages/tarlz', 'save_request_date': '2021-09-29T10:08:00.758586+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298436, 'origin_url': 'https://git.progress-linux.org/users/daniel.baumann/debian/packages/lzip', 'save_request_date': '2021-09-29T10:08:00.976973+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298437, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/takin-data.git', 'save_request_date': '2021-09-29T10:08:01.213923+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298439, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tastools.git', 'save_request_date': '2021-09-29T10:08:01.458344+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298440, 'origin_url': 'https://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils', 'save_request_date': '2021-09-29T10:08:01.671555+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298441, 'origin_url': 'https://c9x.me/git/qbe.git', 'save_request_date': '2021-09-29T10:08:01.891677+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298442, 'origin_url': 'https://git.tuxfamily.org/ostorybook/code.git', 'save_request_date': '2021-09-29T10:08:02.096468+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298444, 'origin_url': 'https://thingshare.ion.nu/thingshare.git', 'save_request_date': '2021-09-29T10:08:02.333379+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298445, 'origin_url': 'https://inqlab.net/git/guile-sodium.git', 'save_request_date': '2021-09-29T10:08:02.603023+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298446, 'origin_url': 'https://git.systemreboot.net/guile-xapian', 'save_request_date': '2021-09-29T10:08:02.980312+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298447, 'origin_url': 'https://git.netfilter.org/iptables', 'save_request_date': '2021-09-29T10:08:03.692976+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298448, 'origin_url': 'https://inqlab.net/git/guile-cbor.git', 'save_request_date': '2021-09-29T10:08:03.966797+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298449, 'origin_url': 'https://git.openembedded.org/openembedded-core', 'save_request_date': '2021-09-29T10:08:04.228102+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298451, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/mcstas.git', 'save_request_date': '2021-09-29T10:08:04.444651+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298437, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/takin-data.git', 'save_request_date': '2021-09-29T10:08:01.213923+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298439, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tastools.git', 'save_request_date': '2021-09-29T10:08:01.458344+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298425, 'origin_url': 'https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/miezetools.git', 'save_request_date': '2021-09-29T10:07:57.811821+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298452, 'origin_url': 'https://git.beyermatthi.as/prometheus-mpd-exporter', 'save_request_date': '2021-09-29T10:08:05.289687+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298453, 'origin_url': 'https://git.joeyh.name/git/arduino-copilot.git', 'save_request_date': '2021-09-29T10:08:05.547289+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298454, 'origin_url': 'https://git.netfilter.org/libmnl', 'save_request_date': '2021-09-29T10:08:05.849481+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298455, 'origin_url': 'https://jff.email/cgit/libunistring.git', 'save_request_date': '2021-09-29T10:08:06.257710+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298456, 'origin_url': 'https://inqlab.net/git/eris.git', 'save_request_date': '2021-09-29T10:08:06.494460+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298457, 'origin_url': 'https://jff.email/cgit/dmidecode.git', 'save_request_date': '2021-09-29T10:08:06.776931+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298697, 'origin_url': 'https://git.tukaani.org/xz.git', 'save_request_date': '2021-09-29T10:08:06.996597+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298458, 'origin_url': 'https://git.dthompson.us/guile-parser-combinators.git', 'save_request_date': '2021-09-29T10:08:07.200536+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298459, 'origin_url': 'http://git.foldling.org/vim-scheme.git', 'save_request_date': '2021-09-29T10:08:07.417546+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298460, 'origin_url': 'https://git.opensvc.com/multipath-tools/.git', 'save_request_date': '2021-09-29T10:08:07.656692+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298461, 'origin_url': 'https://git.osmocom.org/rtl-sdr', 'save_request_date': '2021-09-29T10:08:08.073457+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298463, 'origin_url': 'http://git.kpe.io/uffi.git', 'save_request_date': '2021-09-29T10:08:08.438223+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298465, 'origin_url': 'http://git.kpe.io/clsql.git', 'save_request_date': '2021-09-29T10:08:09.662233+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298466, 'origin_url': 'http://git.kpe.io/ptester.git', 'save_request_date': '2021-09-29T10:08:11.056976+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298467, 'origin_url': 'https://git.osmocom.org/gr-osmosdr', 'save_request_date': '2021-09-29T10:08:11.892136+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298468, 'origin_url': 'https://git.osmocom.org/libosmo-dsp', 'save_request_date': '2021-09-29T10:08:12.314649+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298469, 'origin_url': 'http://git.kpe.io/xlunit.git', 'save_request_date': '2021-09-29T10:08:12.602773+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298470, 'origin_url': 'http://git.kpe.io/kmrcl.git', 'save_request_date': '2021-09-29T10:08:13.082206+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298471, 'origin_url': 'https://git.open-music-kontrollers.ch/lv2/lv2lint', 'save_request_date': '2021-09-29T10:08:13.630219+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298472, 'origin_url': 'https://git.systemreboot.net/exiftool.el', 'save_request_date': '2021-09-29T10:08:14.011802+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298473, 'origin_url': 'http://git.kpe.io/cl-base64.git', 'save_request_date': '2021-09-29T10:08:14.521164+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298475, 'origin_url': 'http://git.savannah.gnu.org/r/gtick.git', 'save_request_date': '2021-09-29T10:08:15.004437+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298476, 'origin_url': 'http://git.sv.gnu.org/r/gtick.git', 'save_request_date': '2021-09-29T10:08:15.279068+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298477, 'origin_url': 'https://git.osmocom.org/gr-iqbal', 'save_request_date': '2021-09-29T10:08:15.475285+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298479, 'origin_url': 'https://git.savannah.gnu.org/r/guix.git', 'save_request_date': '2021-09-29T10:08:15.755672+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298480, 'origin_url': 'https://www.cl.cam.ac.uk/~mgk25/git/tripos', 'save_request_date': '2021-09-29T10:08:16.222379+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298481, 'origin_url': 'https://www.cl.cam.ac.uk/~mgk25/git/jbigkit', 'save_request_date': '2021-09-29T10:08:16.425781+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298482, 'origin_url': 'https://git.rockbox.org/cgit/sandbox.git', 'save_request_date': '2021-09-29T10:08:16.631493+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298483, 'origin_url': 'https://git.rockbox.org/cgit/themesite.git', 'save_request_date': '2021-09-29T10:08:16.856420+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298484, 'origin_url': 'https://git.rockbox.org/cgit/translate.git', 'save_request_date': '2021-09-29T10:08:17.059833+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298485, 'origin_url': 'https://git.rockbox.org/cgit/www.git', 'save_request_date': '2021-09-29T10:08:17.277452+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298486, 'origin_url': 'https://git.rockbox.org/cgit/rockbox.git', 'save_request_date': '2021-09-29T10:08:17.453703+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298487, 'origin_url': 'https://git.qsomula.top/SUGAR__STRAWBERRIES/SUGAR__AND__STRAWBERRIES.git', 'save_request_date': '2021-09-29T10:08:17.696238+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298488, 'origin_url': 'https://source.netsurf-browser.org/alphagen.git', 'save_request_date': '2021-09-29T10:08:17.905282+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298489, 'origin_url': 'https://source.netsurf-browser.org/ttf2f.git', 'save_request_date': '2021-09-29T10:08:18.124307+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298490, 'origin_url': 'https://source.netsurf-browser.org/toolchains.git', 'save_request_date': '2021-09-29T10:08:18.332149+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298491, 'origin_url': 'https://source.netsurf-browser.org/squeeze.git', 'save_request_date': '2021-09-29T10:08:18.541807+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298492, 'origin_url': 'https://source.netsurf-browser.org/packaging/fedora.git', 'save_request_date': '2021-09-29T10:08:18.775088+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298493, 'origin_url': 'https://source.netsurf-browser.org/packaging/debian.git', 'save_request_date': '2021-09-29T10:08:18.958225+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298494, 'origin_url': 'https://source.netsurf-browser.org/nstheme.git', 'save_request_date': '2021-09-29T10:08:19.163989+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298495, 'origin_url': 'https://source.netsurf-browser.org/nsgenbind.git', 'save_request_date': '2021-09-29T10:08:19.383856+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298496, 'origin_url': 'https://source.netsurf-browser.org/netsurf.git', 'save_request_date': '2021-09-29T10:08:19.593296+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298497, 'origin_url': 'https://source.netsurf-browser.org/netsurf-wiki.git', 'save_request_date': '2021-09-29T10:08:19.800945+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298498, 'origin_url': 'https://source.netsurf-browser.org/netsurf-website.git', 'save_request_date': '2021-09-29T10:08:19.997824+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298500, 'origin_url': 'https://source.netsurf-browser.org/netsurf-test.git', 'save_request_date': '2021-09-29T10:08:20.747537+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298501, 'origin_url': 'https://source.netsurf-browser.org/netsurf-all.git', 'save_request_date': '2021-09-29T10:08:20.948499+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298502, 'origin_url': 'https://source.netsurf-browser.org/makerun.git', 'save_request_date': '2021-09-29T10:08:21.168633+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298503, 'origin_url': 'https://source.netsurf-browser.org/libwapcaplet.git', 'save_request_date': '2021-09-29T10:08:21.391437+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298504, 'origin_url': 'https://source.netsurf-browser.org/libutf8proc.git', 'save_request_date': '2021-09-29T10:08:21.575900+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298506, 'origin_url': 'https://source.netsurf-browser.org/libsvgtiny.git', 'save_request_date': '2021-09-29T10:08:21.766147+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298507, 'origin_url': 'https://source.netsurf-browser.org/librufl.git', 'save_request_date': '2021-09-29T10:08:21.971342+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298508, 'origin_url': 'https://source.netsurf-browser.org/librosprite.git', 'save_request_date': '2021-09-29T10:08:22.163567+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298509, 'origin_url': 'https://source.netsurf-browser.org/libpencil.git', 'save_request_date': '2021-09-29T10:08:22.373564+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298510, 'origin_url': 'https://source.netsurf-browser.org/libparserutils.git', 'save_request_date': '2021-09-29T10:08:22.544880+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298511, 'origin_url': 'https://source.netsurf-browser.org/libnsutils.git', 'save_request_date': '2021-09-29T10:08:22.742637+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298512, 'origin_url': 'https://source.netsurf-browser.org/libnspsl.git', 'save_request_date': '2021-09-29T10:08:22.962451+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298513, 'origin_url': 'https://source.netsurf-browser.org/libnspdf.git', 'save_request_date': '2021-09-29T10:08:23.146196+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298514, 'origin_url': 'https://source.netsurf-browser.org/libnslog.git', 'save_request_date': '2021-09-29T10:08:23.341888+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298515, 'origin_url': 'https://source.netsurf-browser.org/libnslayout.git', 'save_request_date': '2021-09-29T10:08:23.541333+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298516, 'origin_url': 'https://source.netsurf-browser.org/libnsgif.git', 'save_request_date': '2021-09-29T10:08:23.747382+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298517, 'origin_url': 'https://source.netsurf-browser.org/libnsfb.git', 'save_request_date': '2021-09-29T10:08:23.950486+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298518, 'origin_url': 'https://source.netsurf-browser.org/libnsbmp.git', 'save_request_date': '2021-09-29T10:08:24.128810+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298519, 'origin_url': 'https://source.netsurf-browser.org/libhubbub.git', 'save_request_date': '2021-09-29T10:08:24.335594+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298520, 'origin_url': 'https://source.netsurf-browser.org/libdom.git', 'save_request_date': '2021-09-29T10:08:24.557136+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298521, 'origin_url': 'https://source.netsurf-browser.org/libcss.git', 'save_request_date': '2021-09-29T10:08:24.760323+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298522, 'origin_url': 'https://source.netsurf-browser.org/json-c.git', 'save_request_date': '2021-09-29T10:08:24.999232+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298523, 'origin_url': 'https://source.netsurf-browser.org/iconv.git', 'save_request_date': '2021-09-29T10:08:25.178058+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298524, 'origin_url': 'https://source.netsurf-browser.org/ci-tools.git', 'save_request_date': '2021-09-29T10:08:25.398196+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298525, 'origin_url': 'https://source.netsurf-browser.org/buildsystem.git', 'save_request_date': '2021-09-29T10:08:25.568154+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298526, 'origin_url': 'https://source.netsurf-browser.org/art.git', 'save_request_date': '2021-09-29T10:08:25.759202+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298527, 'origin_url': 'http://galexander.org/git/simplesshd.git', 'save_request_date': '2021-09-29T10:08:26.002317+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298528, 'origin_url': 'https://tinc-vpn.org/git/tinc', 'save_request_date': '2021-09-29T10:08:26.228797+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298529, 'origin_url': 'https://git.joeyh.name/git/concurrent-output.git', 'save_request_date': '2021-09-29T10:08:26.428783+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298530, 'origin_url': 'https://git.joeyh.name/git/brainfuck-monad.git', 'save_request_date': '2021-09-29T10:08:26.652730+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298532, 'origin_url': 'https://git.neodarz.net/pro/umosapicpp.git', 'save_request_date': '2021-09-29T10:08:26.890221+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298533, 'origin_url': 'https://git.neodarz.net/pro/umosapi.git', 'save_request_date': '2021-09-29T10:08:27.129709+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298534, 'origin_url': 'https://git.neodarz.net/neodarz/waveshare-7inch-touchscreen-driver.git', 'save_request_date': '2021-09-29T10:08:27.295848+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298535, 'origin_url': 'https://git.neodarz.net/neodarz/scripts/feed_finder.git', 'save_request_date': '2021-09-29T10:08:27.489133+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298536, 'origin_url': 'https://git.neodarz.net/neodarz/scripts/convert.py.git', 'save_request_date': '2021-09-29T10:08:27.669389+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298537, 'origin_url': 'https://git.neodarz.net/neodarz/scripts/bandcamp-dl_script.git', 'save_request_date': '2021-09-29T10:08:27.843411+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298538, 'origin_url': 'https://git.neodarz.net/neodarz/scripts/arte.tv-dl.git', 'save_request_date': '2021-09-29T10:08:28.021717+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298539, 'origin_url': 'https://git.neodarz.net/neodarz/scripts/archlinux_iso_checker.git', 'save_request_date': '2021-09-29T10:08:28.184597+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298540, 'origin_url': 'https://git.neodarz.net/neodarz/scripts/SimplyQrGen.git', 'save_request_date': '2021-09-29T10:08:28.359197+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298541, 'origin_url': 'https://git.neodarz.net/neodarz/scripts/EphermalVPN.git', 'save_request_date': '2021-09-29T10:08:28.549753+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298542, 'origin_url': 'https://git.neodarz.net/neodarz/school/the_transcriber_backend.git', 'save_request_date': '2021-09-29T10:08:28.711372+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298543, 'origin_url': 'https://git.neodarz.net/neodarz/resumejson_converter.git', 'save_request_date': '2021-09-29T10:08:28.875127+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298544, 'origin_url': 'https://git.neodarz.net/code_archive/freetorrent.git', 'save_request_date': '2021-09-29T10:08:29.058812+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298545, 'origin_url': 'https://git.e2factory.org/git/e2factory.git', 'save_request_date': '2021-09-29T10:08:29.235115+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298546, 'origin_url': 'https://git.zap.org.au/git/console-fonts-utils.git', 'save_request_date': '2021-09-29T10:08:29.485093+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298547, 'origin_url': 'https://git.zap.org.au/git/console-fonts-lucida.git', 'save_request_date': '2021-09-29T10:08:29.705671+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298548, 'origin_url': 'https://git.zap.org.au/git/console-fonts-zap.git', 'save_request_date': '2021-09-29T10:08:29.896133+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298549, 'origin_url': 'https://git.zap.org.au/git/trader.git', 'save_request_date': '2021-09-29T10:08:30.119825+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'} {'loading_task_id': 400298550, 'origin_url': 'http://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/miezetools.git', 'save_request_date': '2021-09-29T10:08:30.279433+00:00', 'save_request_status': 'accepted', 'save_task_status': 'not yet scheduled', 'visit_date': None, 'visit_status': None, 'visit_type': 'git'}
We will use the following script to create new save requests for those origins:
import os from pprint import pprint
This is the result of the above script execution:
anlambert@carnavalet:~/tmp$ python3 get_save_code_now_dumb_git_repos.py 2>/dev/null https://git.pengutronix.de/cgit/tools/libsocketcan https://git.stoutner.com/UltraPrivacy.git https://git.stoutner.com/UltraList.git https://git.stoutner.com/PrivacyCell.git https://git.stoutner.com/PrivacyBrowserROMIntegration.git https://git.stoutner.com/PrivacyBrowser.git https://git.raptorengineering.com/git/hqemu/ https://forge.touhey.org/casio/packaging.git https://forge.touhey.org/casio/libcasio.git https://git.tuxfamily.org/harmonist/harmonist.git https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tlibs.git https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/miezetools.git https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tlibs.git/ https://depp.brause.cc/nov.el.git https://www.6809.org.uk/git/asm6809.git http://git.ghostscript.com/mupdf.git https://git.joeyh.name/git/oldusenet.git https://git.drobilla.net/ganv.git https://genome-source.gi.ucsc.edu/samtabix.git http://software.legiasoft.com/git/quickcheck.git https://git.progress-linux.org/users/daniel.baumann/debian/packages/lziprecover https://git.progress-linux.org/users/daniel.baumann/debian/packages/plzip https://git.progress-linux.org/users/daniel.baumann/debian/packages/tarlz https://git.progress-linux.org/users/daniel.baumann/debian/packages/lzip https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/takin-data.git https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tastools.git https://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils https://c9x.me/git/qbe.git https://git.tuxfamily.org/ostorybook/code.git https://thingshare.ion.nu/thingshare.git/ https://inqlab.net/git/guile-sodium.git https://git.systemreboot.net/guile-xapian https://git.netfilter.org/iptables https://inqlab.net/git/guile-cbor.git https://git.openembedded.org/openembedded-core https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/mcstas.git/ https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/takin-data.git/ https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/tastools.git/ https://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/miezetools.git/ https://git.beyermatthi.as/prometheus-mpd-exporter https://git.joeyh.name/git/arduino-copilot.git https://git.netfilter.org/libmnl/ https://jff.email/cgit/libunistring.git https://inqlab.net/git/eris.git https://jff.email/cgit/dmidecode.git https://git.tukaani.org/xz.git https://git.dthompson.us/guile-parser-combinators.git http://git.foldling.org/vim-scheme.git https://git.opensvc.com/multipath-tools/.git https://git.osmocom.org/rtl-sdr http://git.kpe.io/uffi.git http://git.kpe.io/clsql.git http://git.kpe.io/ptester.git https://git.osmocom.org/gr-osmosdr https://git.osmocom.org/libosmo-dsp http://git.kpe.io/xlunit.git http://git.kpe.io/kmrcl.git https://git.open-music-kontrollers.ch/lv2/lv2lint https://git.systemreboot.net/exiftool.el http://git.kpe.io/cl-base64.git http://git.savannah.gnu.org/r/gtick.git http://git.sv.gnu.org/r/gtick.git https://git.osmocom.org/gr-iqbal https://git.savannah.gnu.org/r/guix.git https://www.cl.cam.ac.uk/~mgk25/git/tripos/ https://www.cl.cam.ac.uk/~mgk25/git/jbigkit https://git.rockbox.org/cgit/sandbox.git/ https://git.rockbox.org/cgit/themesite.git/ https://git.rockbox.org/cgit/translate.git/ https://git.rockbox.org/cgit/www.git/ https://git.rockbox.org/cgit/rockbox.git/ https://git.qsomula.top/SUGAR__STRAWBERRIES/SUGAR__AND__STRAWBERRIES.git https://source.netsurf-browser.org/alphagen.git/ https://source.netsurf-browser.org/ttf2f.git/ https://source.netsurf-browser.org/toolchains.git/ https://source.netsurf-browser.org/squeeze.git/ https://source.netsurf-browser.org/packaging/fedora.git/ https://source.netsurf-browser.org/packaging/debian.git/ https://source.netsurf-browser.org/nstheme.git/ https://source.netsurf-browser.org/nsgenbind.git/ https://source.netsurf-browser.org/netsurf.git/ https://source.netsurf-browser.org/netsurf-wiki.git/ https://source.netsurf-browser.org/netsurf-website.git/ https://source.netsurf-browser.org/netsurf-test.git/ https://source.netsurf-browser.org/netsurf-all.git/ https://source.netsurf-browser.org/makerun.git/ https://source.netsurf-browser.org/libwapcaplet.git/ https://source.netsurf-browser.org/libutf8proc.git/ https://source.netsurf-browser.org/libsvgtiny.git/ https://source.netsurf-browser.org/librufl.git/ https://source.netsurf-browser.org/librosprite.git/ https://source.netsurf-browser.org/libpencil.git/ https://source.netsurf-browser.org/libparserutils.git/ https://source.netsurf-browser.org/libnsutils.git/ https://source.netsurf-browser.org/libnspsl.git/ https://source.netsurf-browser.org/libnspdf.git/ https://source.netsurf-browser.org/libnslog.git/ https://source.netsurf-browser.org/libnslayout.git/ https://source.netsurf-browser.org/libnsgif.git/ https://source.netsurf-browser.org/libnsfb.git/ https://source.netsurf-browser.org/libnsbmp.git/ https://source.netsurf-browser.org/libhubbub.git/ https://source.netsurf-browser.org/libdom.git/ https://source.netsurf-browser.org/libcss.git/ https://source.netsurf-browser.org/json-c.git/ https://source.netsurf-browser.org/iconv.git/ https://source.netsurf-browser.org/ci-tools.git/ https://source.netsurf-browser.org/buildsystem.git/ https://source.netsurf-browser.org/art.git/ http://galexander.org/git/simplesshd.git https://tinc-vpn.org/git/tinc https://git.joeyh.name/git/concurrent-output.git https://git.joeyh.name/git/brainfuck-monad.git https://git.neodarz.net/pro/umosapicpp.git https://git.neodarz.net/pro/umosapi.git https://git.neodarz.net/neodarz/waveshare-7inch-touchscreen-driver.git https://git.neodarz.net/neodarz/scripts/feed_finder.git https://git.neodarz.net/neodarz/scripts/convert.py.git https://git.neodarz.net/neodarz/scripts/bandcamp-dl_script.git https://git.neodarz.net/neodarz/scripts/arte.tv-dl.git https://git.neodarz.net/neodarz/scripts/archlinux_iso_checker.git https://git.neodarz.net/neodarz/scripts/SimplyQrGen.git https://git.neodarz.net/neodarz/scripts/EphermalVPN.git https://git.neodarz.net/neodarz/school/the_transcriber_backend.git https://git.neodarz.net/neodarz/resumejson_converter.git https://git.neodarz.net/code_archive/freetorrent.git https://git.e2factory.org/git/e2factory.git https://git.zap.org.au/git/console-fonts-utils.git https://git.zap.org.au/git/console-fonts-lucida.git https://git.zap.org.au/git/console-fonts-zap.git https://git.zap.org.au/git/trader.git http://forge.frm2.tum.de/cgit/cgit.cgi/frm2/mira/miezetools.git/
Afterwards, we use the following script to identify the dumb git origins that can be reloaded:
import io import os
Then we copy that dump locally and load it into our development database:
anlambert@carnavalet:~/tmp$ scp anlambert@moma:swh-web-db.json . swh-web-db.json 100% 33MB 5.6MB/s 00:05 anlambert@carnavalet:~/tmp$ django-admin loaddata --settings swh.web.settings.development swh-web-db.json Installed 93893 object(s) from 1 fixture(s) anlambert@carnavalet:~/tmp$
First we need a JSON dump of production database:
anlambert@carnavalet:~/tmp$ ssh anlambert@moma Linux moma 5.10.0-0.bpo.5-amd64 #1 SMP Debian 5.10.24-1~bpo10+1 (2021-03-29) x86_64
Sep 20 2021
Sep 17 2021
Sep 8 2021
Indeed some save requests are still marked as running while they clearly are not.
Sep 3 2021
I've opened a task to try and monitor this pattern [1].
I actually requested https://github.com/tue-alga/CoordinatedSchematization which did
not start for over 45 minutes or so (which is unusual), then noticed that the one
above was also in "scheduled" state over hours and decided to open a task here.
Thanks for the quick help!
I actually requested https://github.com/tue-alga/CoordinatedSchematization which did not start for over 45 minutes or so (which is unusual), then noticed that the one above was also in "scheduled" state over hours and decided to open a task here.
For some unknown reason, that particular origins is scheduled but I don't see any log about that run.
I triggered it back and it got finished almost immediately (uneventful).
(was looking into it)
I'd say this one qualifies (in UTC+2):
@mdidas what are the repositories you requested?
Sep 1 2021
The scheduler is getting there.
We are now able to trigger a runner for that part:
Aug 4 2021
Jun 28 2021
Deployed.
Jun 25 2021
Build is green
Adapt according to review
Build is green
Looks good to me ! Thanks, the save code now handling code definitely needed some cleanup after all those years.
Build has FAILED
Build was aborted
Jun 24 2021
The issue mentioned ^ has been fixed and deployed.
Everything is deployed.
Jun 21 2021
I agree having access to the logs of the task (more or less) in real-time would be very handy (as one can expect on any CI-like tool nowadays).
Jun 18 2021
Jun 16 2021
Remain the runner to actually consume those regularly to be deployed (or maybe it's already the case, i need to check that part).
The first part got deployed (modification in the webapp routines to update the save code
now statuses).
Jun 15 2021
Thanks @ardumont for following up to this task.