To be able to compare the relative performance of different scheduler implementations and policies, among one another as well as over time, we need to actually define, and implement, a set of performance metrics for the scheduler(s).
|Open||olasd||T2345 Improve handling of recurrent loading tasks in scheduler|
|Work in Progress||olasd||T2974 Define (and implement) scheduler performance metrics|
Some potentially interesting and "easy" metrics:
- "origin coverage": Number of origins that have never been visited (lower is better)
- "visit usefulness": Number of uneventful visits per unit of time (lower is better)
- "origin lag": (average?) Time between last origin activity and loading time (lower is better)
More speculative metrics:
- optimizing for "visit regularity/smoothness/smallness": sum of the square (or any other supra-linear function) of the duration of each visit (by unit of time); in combination with the "visit usefulness" metric, this could give us a sense of the amount we're lagging behind any given origin: a shorter (eventful) visit means that the archive is lagging less behind any given origin.