Unstuck running save code now origins
Closed, MigratedEdits Locked
Actions

Assigned To

Authored By

	ardumont
	Jan 13 2022, 11:34 AM

Description

The origins whose status is running (and scheduled a long time ago) may be stuck, we need to reschedule their associated task id:

11:36:59 swh-web@belvedere:5432=> select request_date, origin_url, status, loading_task_status, loading_task_id from save_origin_request where loading_task_status='running' and request_date < now() - interval '1 month';
select request_date, origin_url, status, loading_task_status, loading_task_id from save_origin_request where loading_task_status='running' and request_date < now() - interval '1 month';
+-------------------------------+--------------------------------------------------------------+----------+---------------------+-----------------+
|         request_date          |                          origin_url                          |  status  | loading_task_status | loading_task_id |
+-------------------------------+--------------------------------------------------------------+----------+---------------------+-----------------+
| 2021-03-22 07:03:38.046+00    | https://github.com/kusl/wgeteveryday                         | accepted | running             |       378218248 |
| 2021-04-01 16:57:50.205+00    | https://scm.gforge.inria.fr/anonscm/git/simty/simty.git      | accepted | running             |       379069991 |
| 2021-04-20 08:52:13.105092+00 | https://github.com/coreutils/gnulib                          | accepted | running             |       380606658 |
| 2021-04-20 11:40:21.612664+00 | https://github.com/unitystation/unitystation                 | accepted | running             |       380617630 |
| 2021-09-26 15:25:37.040968+00 | http://svn.code.sf.net/p/sauerbraten/code                    | accepted | running             |       400154016 |
| 2021-10-19 11:43:55.370658+00 | https://gitlab.com/inkscape/inkscape                         | accepted | running             |       401116302 |
| 2021-07-17 22:43:29.870783+00 | https://github.com/keybase/client                            | accepted | running             |       396435047 |
| 2021-05-15 06:09:05.96959+00  | https://git.libreoffice.org/translations/                    | accepted | running             |       381569600 |
| 2021-05-23 12:18:02.213604+00 | https://git.savannah.gnu.org/git/gnulib.git                  | accepted | running             |       381570771 |
| 2021-07-26 12:59:50.038714+00 | https://svn.r-project.org/R-dev-web/trunk/                   | accepted | running             |       396985198 |
| 2021-07-26 12:54:25.784339+00 | svn://svn.code.sf.net/p/codeblocks/code/trunk                | accepted | running             |       396984987 |
| 2021-08-10 09:31:51.290321+00 | https://github.com/CocoaPods/Specs                           | accepted | running             |       397492293 |
| 2021-08-21 08:06:47.263554+00 | https://github.com/404-not-find/client                       | accepted | running             |       398036169 |
| 2021-10-22 13:32:29.289308+00 | http://floppsie.comp.glam.ac.uk/gm2                          | accepted | running             |       401265256 |
| 2021-06-24 18:56:00.441183+00 | https://anonhg.netbsd.org/pkgsrc-public/                     | accepted | running             |       381575556 |
| 2021-06-24 18:55:55.386673+00 | https://anonhg.netbsd.org/pkgsrc-draft/                      | accepted | running             |       381575557 |
| 2021-06-24 18:55:51.53746+00  | https://anonhg.netbsd.org/pkgsrc/                            | accepted | running             |       381575558 |
| 2021-06-24 18:55:41.589663+00 | https://anonhg.netbsd.org/src-public/                        | accepted | running             |       381575560 |
| 2021-06-24 18:55:35.63743+00  | https://anonhg.netbsd.org/src-draft/                         | accepted | running             |       381575561 |
| 2021-06-24 16:53:31.099868+00 | git://git.archlinux.org/svntogit/community.git               | accepted | running             |       381575242 |
| 2021-06-22 21:56:29.522544+00 | https://github.com/jlippold/tweakCompatible                  | accepted | running             |       381575019 |
| 2021-06-29 17:08:37.262648+00 | git://git.archlinux.org/svntogit/packages.git                | accepted | running             |       381575775 |
| 2021-09-09 20:13:05.170087+00 | https://source.puri.sm/Librem5/linux-next.git                | accepted | running             |       399230899 |
| 2020-01-24 00:29:45.32+00     | https://github.com/CambridgeSemiticsLab/BH_time_collocations | accepted | running             |       269843604 |
| 2021-11-10 21:02:01.294182+00 | https://git.texlive.info/texlive                             | accepted | running             |       402275446 |
| 2021-12-10 14:02:24.992009+00 | https://github.com/godotengine/godot                         | accepted | running             |       403530429 |
+-------------------------------+--------------------------------------------------------------+----------+---------------------+-----------------+
(26 rows)

Time: 255.180 ms

Related Objects

Mentioned Here: P1258 (An Untitled Masterwork)

Event Timeline

ardumont renamed this task from Unstuck running save code now origin to Unstuck running save code now origins.Jan 13 2022, 11:34 AM

ardumont triaged this task as High priority.

ardumont created this task.

ardumont updated the task description. (Show Details)

ardumont updated the task description. (Show Details)Jan 13 2022, 11:38 AM

ardumont updated the task description. (Show Details)

ardumont changed the task status from Open to Work in Progress.Jan 17 2022, 12:22 PM

Those tasks were updated with a status failed [1] [2]
Their associated scheduler task id where archived recently [3]
They have been rescheduled through the save code now cli [4]
Their ingestion is ongoing and their associated status should update once done

[1] Those were most likely oom killed since there has been no update on their status.

[2]

12:05:17 swh-web@belvedere:5432=> update save_origin_request set loading_task_status='failed' where id in (71357, 74433, 75668, 92802, 95563, 86401, 78750, 79867, 87538, 87536, 88780, 89569, 95921, 84095, 84094, 84093, 84091, 84090, 84080, 83883, 84583, 91859,  6930, 98106, 01491, 02346);
UPDATE 26
Time: 60.416 ms

[3]

13:48:58 softwareheritage-scheduler@belvedere:5432=> select * from task where id in (378218248, 379069991, 380606658, 380617630, 400154016, 401116302, 396435047, 381569600, 381570771, 396985198, 396984987, 397492293, 398036169, 401265256, 381575556, 381575557, 381575558, 381575560, 381575561, 381575242, 381575019, 381575775, 399230899, 269843604, 402275446, 403530429);
(0 rows)

[4]

$ cat save-code-now.csv | swh web --config-file ~/.config/swh/global.yml save submit-request
[{"origin_url": "https://scm.gforge.inria.fr/anonscm/git/simty/simty.git", "visit_type": "git", "save_request_date": "2022-01-17T11:32:18.234774+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069622, "note": null}, {"origin_url": "https://github.com/unitystation/unitystation", "visit_type": "git", "save_request_date": "2022-01-17T11:32:18.784397+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069623, "note": null}, {"origin_url": "http://svn.code.sf.net/p/sauerbraten/code", "visit_type": "svn", "save_request_date": "2022-01-17T11:32:19.417732+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069624, "note": null}, {"origin_url": "https://gitlab.com/inkscape/inkscape", "visit_type": "git", "save_request_date": "2022-01-17T11:32:19.767342+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069625, "note": null}, {"origin_url": "https://github.com/keybase/client", "visit_type": "git", "save_request_date": "2022-01-17T11:32:20.202335+00:00", "save_request_status": "accepted", "save_task_status": "running", "visit_status": "created", "visit_date": "2022-01-17T11:32:21.457692+00:00", "loading_task_id": 405069626, "note": null}, {"origin_url": "https://git.libreoffice.org/translations", "visit_type": "git", "save_request_date": "2022-01-17T11:32:24.200399+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069629, "note": null}, {"origin_url": "https://git.savannah.gnu.org/git/gnulib.git", "visit_type": "git", "save_request_date": "2022-01-17T11:32:24.551241+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069631, "note": null}, {"origin_url": "https://svn.r-project.org/R-dev-web/trunk", "visit_type": "svn", "save_request_date": "2022-01-17T11:32:25.119235+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069632, "note": null}, {"origin_url": "svn://svn.code.sf.net/p/codeblocks/code/trunk", "visit_type": "svn", "save_request_date": "2022-01-17T11:32:25.331133+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069633, "note": null}, {"origin_url": "https://github.com/CocoaPods/Specs", "visit_type": "git", "save_request_date": "2022-01-17T11:32:25.696584+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069635, "note": null}, {"origin_url": "https://github.com/404-not-find/client", "visit_type": "git", "save_request_date": "2022-01-17T11:32:29.661549+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069638, "note": null}, {"origin_url": "http://floppsie.comp.glam.ac.uk/gm2", "visit_type": "git", "save_request_date": "2022-01-17T11:32:30.554341+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069640, "note": null}, {"origin_url": "https://anonhg.netbsd.org/pkgsrc-public", "visit_type": "hg", "save_request_date": "2022-01-17T11:32:30.915497+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069641, "note": null}, {"origin_url": "https://anonhg.netbsd.org/pkgsrc-draft", "visit_type": "hg", "save_request_date": "2022-01-17T11:32:31.414971+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069642, "note": null}, {"origin_url": "https://anonhg.netbsd.org/pkgsrc", "visit_type": "hg", "save_request_date": "2022-01-17T11:32:31.621251+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069643, "note": null}, {"origin_url": "https://anonhg.netbsd.org/src-public", "visit_type": "hg", "save_request_date": "2022-01-17T11:32:31.785305+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069644, "note": null}, {"origin_url": "https://anonhg.netbsd.org/src-draft", "visit_type": "hg", "save_request_date": "2022-01-17T11:32:32.088883+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069645, "note": null}, {"origin_url": "git://git.archlinux.org/svntogit/community.git", "visit_type": "git", "save_request_date": "2022-01-17T11:32:32.592602+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069646, "note": null}, {"origin_url": "https://github.com/jlippold/tweakCompatible", "visit_type": "git", "save_request_date": "2022-01-17T11:32:33.224794+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069647, "note": null}, {"origin_url": "git://git.archlinux.org/svntogit/packages.git", "visit_type": "git", "save_request_date": "2022-01-17T11:32:34.805022+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069650, "note": null}, {"origin_url": "https://source.puri.sm/Librem5/linux-next.git", "visit_type": "git", "save_request_date": "2022-01-17T11:32:35.713263+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069652, "note": null}, {"origin_url": "https://github.com/CambridgeSemiticsLab/BH_time_collocations", "visit_type": "git", "save_request_date": "2022-01-17T11:32:36.623927+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069653, "note": null}, {"origin_url": "https://git.texlive.info/texlive", "visit_type": "git", "save_request_date": "2022-01-17T11:32:37.568723+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069654, "note": null}, {"origin_url": "https://github.com/godotengine/godot", "visit_type": "git", "save_request_date": "2022-01-17T11:32:37.753028+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069656, "note": null}, {"origin_url": "https://plugins.svn.wordpress.org/wp-activity-log-for-woocommerce", "visit_type": "svn", "save_request_date": "2022-01-17T11:32:39.873069+00:00", "save_request_status": "accepted", "save_task_status": "not yet scheduled", "visit_status": null, "visit_date": null, "loading_task_id": 405069657, "note": null}]

[1] P1258: save-code-now.csv

Remains 8 origins on running state as they are currently being ingested (and those are large origins).

I'm done with what's possible to do there so closing.

This task has been migrated to GitLab.

Unstuck running save code now originsClosed, MigratedEdits LockedActions

Description

Related Objects

Event Timeline

Unstuck running save code now origins
Closed, MigratedEdits Locked
Actions