softwareheritage=> select count(*) from origin_visit where status = 'ongoing' and date < date(now()) - 365 ; count ------- 65189 (1 row)
As shown above, there is a number of stale visits in the scheduler DB: in the example we have 65k visits that are marked as still ongoing, even if they started more than 1 year ago.
We should clean them up.
A related question is how to avoid they remain around on an ongoing basis…
We should clean them up.
done
In terms of time, it was mostly during the 2016-2017 time period, when loaders could get killed because they were using too much memory (thus not finishing properly).
This is no longer the case.
I should expect that ongoing status to be on a constant level (sum of current visits done by loaders).
I'll check back again later to ensure this is the case.
That could be one interesting metric to keep on track.
Raising an alert if that one goes offtrack.