The current performance of the directory backfiller is not great (the ETA at the current rate is one or two years).
This isn't too surprising as the current code is very naive: it processes directories in hash order, and for each directory it does a massive join to get the directory entries.
There's a few suggestions to improve this:
- @douardda suggested processing directories in insertion order (~ the infamous internal object_id field), which may give us better performance due to cache effects (on the, shared, directory_entry_* tables).
- We could just try using the new, more performant database server and see whether the ETA is more sensible.