This diff consists in a stack of revisions that refactor the pytest_plugin so
that:
- the kafka_server fixture ensures topics exist on the broker beforehand,
- the kafka_prefix is used as is everywhere to prevent some naughty code coupling (user of the plugin *must* also modify the kafka_prefix the same way it was done here).
Included revisions are:
- pytest_plugin: improve error message in consume_messages() helper function
- pytest_plugin: do not crash if the list of consumed messages is empty
if assert_all_objects_consumed() helper function.
- pytest_plugin: modify the kafka_server fixture so topics are created beforehand
Since we now check that topics exist on the kafka broker when instanciating a JournalClient, topics but be created when the mock kafka broker is started for tests to keep working (e.g. in swh.storage).
This needs a small adaptation in the JournalClient code itself to ensure "empty" messages used to create topics will be silently ignored.
The kafka_server() fixture is split in 2 fixtures:- kafka_server_base() that only creates the mock kafka server,
- kafka_server that() uses this later and creates the topics resulting from the conjunction of the kafka_prefix() and the new object_types() fixture.
This new object_types() fixture is thus used to define the list of topics (in conjunction with the kafka_prefix fixture) a (mock) kafka_server() will create at startup time.
- test: replace usage of a strategy .example() by a hardcoded value
to prevent a warning from hypothesis.
- pytest_plugin: remove '.swh.journal.objects' from test_config fixture
This induces a tigh coupling with code from other packages that use this fixture (e.g. swh.storage currently have to modify the kafka_prefix resulting from the fixture so the consumer and the writer are in sync).
If someone wants a customized version of a kafka_prefix, it must be done by overriding the kafka_prefix fixture, not by modifying its returned value.
Note that this commit will break swh.storage.tests.test_kafka_writer until it is updated.
- pytest_plugin: use the object_types fixture in test_config
instead of hardcoding a specific list there that may become out of sync with the default list of topics used elsewhere in the pytest_plugin.
Depends on D3126.