this is necessary to ensure these messages are committed in kafka,
otherwise, since the (considered) empty partition is unsubscribed from,
it never gets committed in JournalClient.handle_messages() (since this
later only commit assigned partitions).
[the second revision in this diff handles that later aspect; not sure
I want to stash these 2 revisions or not...]
Ensure offset are committed ony after worker_fn is executed without tb
this requires to overload the JournalClient.handle_messages() method in
JournalClientOffsetRanges to make sure "pending" messages are
committed after the proper execution of worker_fn.
Doing so, we can both unsubscribe from "oef" partitions on the fly (with
"oef" meaning when the partition has been consumed up to the high
watermark offset at the beginning of the export), and commit ALL offsets
that needs to be but only after proper execution of the worker_fn
This should guarantee proper and consistent behavior (famous last
Depends on D6234