Add a specific cooldown for notfound origins

This allows us to avoid repeating visits on them, until a next pass of
the lister can mark them as disabled.