Page MenuHomeSoftware Heritage

Fix compatibility with hypothesis >= 5.49.0.
AbandonedPublic

Authored by vlorentz on Jan 12 2021, 1:26 PM.

Details

Reviewers
None
Group Reviewers
Reviewers
Summary

It errored with:

the 'swh_storage' fixture, which is reset between function calls but
not between test cases generated by `@given(...)`.

Diff Detail

Event Timeline

Build has FAILED

Patch application report for D4842 (id=17141)

Rebasing onto 2b35198d30...

First, rewinding head to replay your work on top of it...
Applying: Fix compatibility with hypothesis >= 5.49.0.
Changes applied before test
commit 486261b29e28b3446d16ac83f5bb56e6e5619431
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jan 12 13:25:04 2021 +0100

    Fix compatibility with hypothesis >= 5.49.0.
    
    It errored with:
    
        the 'swh_storage' fixture, which is reset between function calls but
        not between test cases generated by `@given(...)`.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1088/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1088/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 12 2021, 1:30 PM
Harbormaster failed remote builds in B18285: Diff 17141!

Build is green

Patch application report for D4842 (id=17143)

Rebasing onto 2b35198d30...

First, rewinding head to replay your work on top of it...
Applying: Fix compatibility with hypothesis >= 5.49.0.
Changes applied before test
commit 7bb878036612524c362d7680a5fa1993cf8ba46e
Author: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Jan 12 13:25:04 2021 +0100

    Fix compatibility with hypothesis >= 5.49.0.
    
    It errored with:
    
        the 'swh_storage' fixture, which is reset between function calls but
        not between test cases generated by `@given(...)`.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1090/ for more details.

I think you could add the suppress_health_check settings directly in the hypothesis profiles for swh-storage, like in swh-web.

Also, I think these changes will break the tests when building the storage debian package as HealthCheck.function_scoped_fixture is a recent addition and the hypothesis version packaged in buster is quite outdated.

For the record, I encountered similar warnings in swh-web and I silented them the following way. This is not the best solution but at least the warnings do not pollute pytest output.

For the record, I encountered similar warnings in swh-web and I silented them the following way. This is not the best solution but at least the warnings do not pollute pytest output.

Oh jeez, they turned the warning into an error. I did not notice at first.

I think you could add the suppress_health_check settings directly in the hypothesis profiles for swh-storage

i don't think it's a good idea; there's a legitimate reason for this warning/error.

Also, I think these changes will break the tests when building the storage debian package as HealthCheck.function_scoped_fixture is a recent addition and the hypothesis version packaged in buster is quite outdated.

Indeed. I don't have a good solution for this. We could have a wrapper for the decorator, but it's ugly :(

swh-web tests are also broken after hypothesis upgrade to 6.0. I think I will check on hypothesis version in conftest.py and add the HealthCheck.function_scoped_fixture in the suppress_health_check of the hypothesis profiles.

FAILED swh/web/tests/api/test_apiresponse.py::test_api_endpoints_have_cors_headers - hypothesis.errors.FailedHealthCheck: swh/web/tests/api/test_apiresponse.py::test_api_endpoints_have_cors_headers uses the ...
FAILED swh/web/tests/api/views/test_graph.py::test_graph_response_resolve_origins - hypothesis.errors.FailedHealthCheck: swh/web/tests/api/views/test_graph.py::test_graph_response_resolve_origins uses the 'm...
FAILED swh/web/tests/api/views/test_identifiers.py::test_swhid_resolve_success - hypothesis.errors.FailedHealthCheck: swh/web/tests/api/views/test_identifiers.py::test_swhid_resolve_success uses the 'client'...
FAILED swh/web/tests/api/views/test_vault.py::test_api_vault_cook - hypothesis.errors.FailedHealthCheck: swh/web/tests/api/views/test_vault.py::test_api_vault_cook uses the 'mocker' fixture, which is reset b...
FAILED swh/web/tests/api/views/test_vault.py::test_api_vault_cook_notfound - hypothesis.errors.FailedHealthCheck: swh/web/tests/api/views/test_vault.py::test_api_vault_cook_notfound uses the 'mocker' fixture...
FAILED swh/web/tests/browse/views/test_content.py::test_content_view_text - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_view_text uses the 'client' fixture, ...
FAILED swh/web/tests/browse/views/test_content.py::test_content_view_text_no_highlight - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_view_text_no_highlight u...
FAILED swh/web/tests/browse/views/test_content.py::test_content_view_no_utf8_text - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_view_no_utf8_text uses the 'c...
FAILED swh/web/tests/browse/views/test_content.py::test_content_view_image - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_view_image uses the 'client' fixture...
FAILED swh/web/tests/browse/views/test_content.py::test_content_view_image_no_rendering - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_view_image_no_rendering...
FAILED swh/web/tests/browse/views/test_content.py::test_content_view_text_with_path - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_view_text_with_path uses th...
FAILED swh/web/tests/browse/views/test_content.py::test_content_raw_text - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_raw_text uses the 'client' fixture, wh...
FAILED swh/web/tests/browse/views/test_content.py::test_content_raw_no_utf8_text - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_raw_no_utf8_text uses the 'cli...
FAILED swh/web/tests/browse/views/test_content.py::test_content_raw_bin - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_raw_bin uses the 'client' fixture, whic...
FAILED swh/web/tests/browse/views/test_content.py::test_content_request_errors - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_request_errors uses the 'client'...
FAILED swh/web/tests/browse/views/test_content.py::test_content_bytes_missing - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_bytes_missing uses the 'client' f...
FAILED swh/web/tests/browse/views/test_content.py::test_content_uppercase - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_uppercase uses the 'client' fixture, ...
FAILED swh/web/tests/browse/views/test_content.py::test_content_utf8_detected_as_binary_display - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_utf8_detected_a...
FAILED swh/web/tests/browse/views/test_content.py::test_content_origin_snapshot_branch_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_origin_snapshot_b...
FAILED swh/web/tests/browse/views/test_content.py::test_content_origin_snapshot_release_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_content.py::test_content_origin_snapshot_...
FAILED swh/web/tests/browse/views/test_directory.py::test_root_directory_view - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_root_directory_view uses the 'client' f...
FAILED swh/web/tests/browse/views/test_directory.py::test_sub_directory_view - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_sub_directory_view uses the 'client' fix...
FAILED swh/web/tests/browse/views/test_directory.py::test_sub_directory_view_origin_context - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_sub_directory_view_origin...
FAILED swh/web/tests/browse/views/test_directory.py::test_directory_request_errors - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_directory_request_errors uses the ...
FAILED swh/web/tests/browse/views/test_directory.py::test_directory_with_invalid_path - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_directory_with_invalid_path use...
FAILED swh/web/tests/browse/views/test_directory.py::test_directory_uppercase - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_directory_uppercase uses the 'client' f...
FAILED swh/web/tests/browse/views/test_directory.py::test_permalink_box_context - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_permalink_box_context uses the 'clien...
FAILED swh/web/tests/browse/views/test_directory.py::test_directory_origin_snapshot_branch_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_directory_origin_sn...
FAILED swh/web/tests/browse/views/test_directory.py::test_content_origin_snapshot_release_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_directory.py::test_content_origin_snaps...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_content_id_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_content_id_browse uses the 'client' f...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_directory_id_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_directory_id_browse uses the 'clien...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_revision_id_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_revision_id_browse uses the 'client'...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_release_id_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_release_id_browse uses the 'client' f...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_snapshot_id_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_snapshot_id_browse uses the 'client'...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_bad_id_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_bad_id_browse uses the 'client' fixture, ...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_content_id_optional_parts_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_content_id_optional_pa...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_origin_id_not_resolvable - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_origin_id_not_resolvable uses ...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_legacy_swhid_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_legacy_swhid_browse uses the 'clien...
FAILED swh/web/tests/browse/views/test_identifiers.py::test_browse_swhid_special_characters_escaping - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_identifiers.py::test_browse_swhid_s...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_visits_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_visits_browse uses the 'client' fixtu...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_content_view - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_content_view uses the 'client' fixture...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_root_directory_view - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_root_directory_view uses the 'c...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_sub_directory_view - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_sub_directory_view uses the 'cli...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_branches - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_branches uses the 'client' fixture, which ...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_releases - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_releases uses the 'client' fixture, which ...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_snapshot_null_branch - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_snapshot_null_branch uses the ...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_snapshot_invalid_branch - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_snapshot_invalid_branch use...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_visits_origin_not_found - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_visits_origin_not_found use...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_origin_directory_no_visit - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_origin_directory_no_visit...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_origin_directory_unknown_visit - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_origin_directory_unk...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_origin_directory_not_found - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_origin_directory_not_fou...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_origin_content_no_visit - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_origin_content_no_visit use...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_origin_content_unknown_visit - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_origin_content_unknown...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_origin_content_directory_empty_snapshot - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_origin_cont...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_origin_content_not_found - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_origin_content_not_found u...
FAILED swh/web/tests/browse/views/test_origin.py::test_browse_directory_snapshot_not_found - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_browse_directory_snapshot_not...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_empty_snapshot - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_empty_snapshot uses the 'client' fix...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_empty_snapshot_null_revision - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_empty_snapshot_null_re...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_release_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_release_browse uses the 'client' fix...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_release_browse_not_found - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_release_browse_not_found u...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_browse_directory_branch_with_non_resolvable_revision - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origi...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_content_no_path - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_content_no_path uses the 'client' f...
FAILED swh/web/tests/browse/views/test_origin.py::test_origin_branches_pagination_with_alias - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_origin.py::test_origin_branches_pagination_...
FAILED swh/web/tests/browse/views/test_release.py::test_release_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_release.py::test_release_browse uses the 'client' fixture, which ...
FAILED swh/web/tests/browse/views/test_release.py::test_release_browse_with_origin_snapshot - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_release.py::test_release_browse_with_origin_...
FAILED swh/web/tests/browse/views/test_release.py::test_release_browse_not_found - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_release.py::test_release_browse_not_found uses the 'cli...
FAILED swh/web/tests/browse/views/test_release.py::test_release_uppercase - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_release.py::test_release_uppercase uses the 'client' fixture, ...
FAILED swh/web/tests/browse/views/test_revision.py::test_revision_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_revision.py::test_revision_browse uses the 'client' fixture, wh...
FAILED swh/web/tests/browse/views/test_revision.py::test_revision_origin_snapshot_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_revision.py::test_revision_origin_snapshot_brow...
FAILED swh/web/tests/browse/views/test_revision.py::test_revision_log_browse - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_revision.py::test_revision_log_browse uses the 'client' fix...
FAILED swh/web/tests/browse/views/test_revision.py::test_revision_request_errors - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_revision.py::test_revision_request_errors uses the 'cli...
FAILED swh/web/tests/browse/views/test_revision.py::test_revision_uppercase - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_revision.py::test_revision_uppercase uses the 'client' fixtu...
FAILED swh/web/tests/browse/views/test_revision.py::test_revision_invalid_path - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_revision.py::test_revision_invalid_path uses the 'client'...
FAILED swh/web/tests/browse/views/test_revision.py::test_revision_metadata_display - hypothesis.errors.FailedHealthCheck: swh/web/tests/browse/views/test_revision.py::test_revision_metadata_display uses the ...
FAILED swh/web/tests/common/test_identifiers.py::test_resolve_swhids_snapshot_context - hypothesis.errors.FailedHealthCheck: swh/web/tests/common/test_identifiers.py::test_resolve_swhids_snapshot_context use...
FAILED swh/web/tests/common/test_identifiers.py::test_resolve_swhid_with_escaped_chars - swh.web.common.exc.NotFoundExc: Origin with url http://example.org/?project=abc; not found!
FAILED swh/web/tests/common/test_middlewares.py::test_exception_middleware_disabled - hypothesis.errors.FailedHealthCheck: swh/web/tests/common/test_middlewares.py::test_exception_middleware_disabled uses th...
FAILED swh/web/tests/common/test_middlewares.py::test_exception_middleware_enabled - hypothesis.errors.FailedHealthCheck: swh/web/tests/common/test_middlewares.py::test_exception_middleware_enabled uses the ...
FAILED swh/web/tests/common/test_origin_visits.py::test_get_origin_visits - hypothesis.errors.FailedHealthCheck: swh/web/tests/common/test_origin_visits.py::test_get_origin_visits uses the 'mocker' fixture, ...
FAILED swh/web/tests/common/test_origin_visits.py::test_get_origin_visit - hypothesis.errors.FailedHealthCheck: swh/web/tests/common/test_origin_visits.py::test_get_origin_visit uses the 'mocker' fixture, wh...
FAILED swh/web/tests/common/test_origin_visits.py::test_get_origin_visit_latest_snapshot - hypothesis.errors.FailedHealthCheck: swh/web/tests/common/test_origin_visits.py::test_get_origin_visit_latest_snapsh...
FAILED swh/web/tests/misc/test_badges.py::test_content_badge - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_content_badge uses the 'client' fixture, which is reset between fun...
FAILED swh/web/tests/misc/test_badges.py::test_directory_badge - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_directory_badge uses the 'client' fixture, which is reset between...
FAILED swh/web/tests/misc/test_badges.py::test_origin_badge - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_origin_badge uses the 'client' fixture, which is reset between funct...
FAILED swh/web/tests/misc/test_badges.py::test_release_badge - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_release_badge uses the 'client' fixture, which is reset between fun...
FAILED swh/web/tests/misc/test_badges.py::test_revision_badge - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_revision_badge uses the 'client' fixture, which is reset between f...
FAILED swh/web/tests/misc/test_badges.py::test_snapshot_badge - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_snapshot_badge uses the 'client' fixture, which is reset between f...
FAILED swh/web/tests/misc/test_badges.py::test_badge_errors - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_badge_errors uses the 'client' fixture, which is reset between funct...
FAILED swh/web/tests/misc/test_badges.py::test_badge_endpoints_have_cors_header - hypothesis.errors.FailedHealthCheck: swh/web/tests/misc/test_badges.py::test_badge_endpoints_have_cors_header uses the 'clien...

swh-web tests are also broken after hypothesis upgrade to 6.0. I think I will check on hypothesis version in conftest.py and add the HealthCheck.function_scoped_fixture in the suppress_health_check of the hypothesis profiles.

D4845

Heads up, D4850 got landed with a restriction on hypothesis < 6 in storage's requirements.