Page MenuHomeSoftware Heritage

deposit: Unstuck buster debian build
Closed, MigratedEdits Locked

Description

As usual, tox and pypi build are fine.

This time the unstable build is fine but the buster build [1] (the one that matters for deployment) fails.

Unstuck it.

[1] https://jenkins.softwareheritage.org/job/debian/job/packages/job/DDEP/job/gbp-buildpackage/94/consoleFull

Event Timeline

ardumont triaged this task as Normal priority.Feb 26 2021, 10:42 AM
ardumont created this task.

django version:

ii  python3-django 1:1.11.29-1~deb10u1 all          High-level Python web development framework (Python 3 version)

Trigger the build:

$ git checkout debian/buster-swh  # stable build branch
$ swh-debian-build-sync           # synchronize the state with jenkins
$ swh-debian-build-stable         # build for stable

The build when failing drops us into a chroot.

There is at least one import failure, steps to reproduce:

(buster-amd64-sbuild)root@yavin4:/build/swh-deposit-4z7JyE# su tony -s /bin/bash
(buster-amd64-sbuild)tony@yavin4:/build/swh-deposit-4z7JyE$ cd swh-deposit-0.10.1/
(buster-amd64-sbuild)tony@yavin4:/build/swh-deposit-4z7JyE/swh-deposit-0.10.1$ python3 -m pytest -x swh
============================================================================================================= test session starts =============================================================================================================
...
>   from django.urls import URLPattern, URLResolver
E   ImportError: cannot import name 'URLPattern' from 'django.urls' (/usr/lib/python3/dist-packages/django/urls/__init__.py)

swh/deposit/urls.py:14: ImportError

It's a typing issue.

After patching swh/deposit/urls to:

try:
    from django.urls import URLPattern, URLResolver
except ImportError:
    # django 1.11
    from django.urls import (
        RegexURLPattern as URLPattern,
        RegexURLResolver as URLResolver,
    )

Tests ran fine:

(buster-amd64-sbuild)tony@yavin4:/build/swh-deposit-4z7JyE/swh-deposit-0.10.1$ emacs /usr/lib/python3/dist-packages/django/urls/__init__.py swh/deposit/urls.py
(failed reverse-i-search)`pytest': emacs /usr/lib/python3/dist-packages/django/urls/__init__.py swh/deposit/urls.py ^C
(buster-amd64-sbuild)tony@yavin4:/build/swh-deposit-4z7JyE/swh-deposit-0.10.1$ ^C
(failed reverse-i-search)`python3 -m ': emacs /usr/lib/^Cthon3/dist-packages/django/urls/__init__.py
(buster-amd64-sbuild)tony@yavin4:/build/swh-deposit-4z7JyE/swh-deposit-0.10.1$ python3 -m pytest -x swh/
============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.7.3, pytest-3.10.1, py-1.7.0, pluggy-0.8.0
Django settings: swh.deposit.settings.testing (from ini file)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/build/swh-deposit-4z7JyE/swh-deposit-0.10.1/.hypothesis/examples')
rootdir: /build/swh-deposit-4z7JyE/swh-deposit-0.10.1, inifile: pytest.ini
plugins: requests-mock-1.5.2, postgresql-2.2.0, mock-1.7.1, django-3.1.2, hypothesis-3.71.11, celery-4.4.2
collected 243 items

swh/deposit/tests/test_common.py .                                                                                                                                                                                                      [  0%]
swh/deposit/tests/test_gunicorn_config.py ...                                                                                                                                                                                           [  1%]
swh/deposit/tests/test_init.py .                                                                                                                                                                                                        [  2%]
swh/deposit/tests/test_utils.py ............................                                                                                                                                                                            [ 13%]
swh/deposit/tests/api/test_checks.py ..........                                                                                                                                                                                         [ 17%]
swh/deposit/tests/api/test_collection.py ...                                                                                                                                                                                            [ 18%]
swh/deposit/tests/api/test_collection_add_to_origin.py .....                                                                                                                                                                            [ 20%]
swh/deposit/tests/api/test_collection_post_atom.py .............................                                                                                                                                                        [ 32%]
swh/deposit/tests/api/test_collection_post_binary.py .........                                                                                                                                                                          [ 36%]
swh/deposit/tests/api/test_collection_post_multipart.py ........                                                                                                                                                                        [ 39%]
swh/deposit/tests/api/test_collection_reuse_slug.py .......                                                                                                                                                                             [ 42%]
swh/deposit/tests/api/test_converters.py .....                                                                                                                                                                                          [ 44%]
swh/deposit/tests/api/test_delete.py .....                                                                                                                                                                                              [ 46%]
swh/deposit/tests/api/test_deposit_list.py ..                                                                                                                                                                                           [ 47%]
swh/deposit/tests/api/test_deposit_private_check.py .............                                                                                                                                                                       [ 53%]
swh/deposit/tests/api/test_deposit_private_read_archive.py ...                                                                                                                                                                          [ 54%]
swh/deposit/tests/api/test_deposit_private_read_metadata.py ......                                                                                                                                                                      [ 56%]
swh/deposit/tests/api/test_deposit_private_update_status.py ......                                                                                                                                                                      [ 59%]
swh/deposit/tests/api/test_deposit_schedule.py ..                                                                                                                                                                                       [ 60%]
swh/deposit/tests/api/test_deposit_state.py ......                                                                                                                                                                                      [ 62%]
swh/deposit/tests/api/test_deposit_update.py ...                                                                                                                                                                                        [ 63%]
swh/deposit/tests/api/test_deposit_update_atom.py ...............                                                                                                                                                                       [ 69%]
swh/deposit/tests/api/test_deposit_update_binary.py ......                                                                                                                                                                              [ 72%]
swh/deposit/tests/api/test_exception.py ..                                                                                                                                                                                              [ 73%]
swh/deposit/tests/api/test_get_file.py ..                                                                                                                                                                                               [ 74%]
swh/deposit/tests/api/test_parsers.py ..                                                                                                                                                                                                [ 74%]
swh/deposit/tests/api/test_service_document.py ....                                                                                                                                                                                     [ 76%]
swh/deposit/tests/cli/test_admin.py ..............                                                                                                                                                                                      [ 82%]
swh/deposit/tests/cli/test_client.py .........................                                                                                                                                                                          [ 92%]
swh/deposit/tests/loader/test_checker.py ...                                                                                                                                                                                            [ 93%]
swh/deposit/tests/loader/test_client.py ............                                                                                                                                                                                    [ 98%]
swh/deposit/tests/loader/test_tasks.py ...                                                                                                                                                                                              [100%]

============================================================================================================== warnings summary ===============================================================================================================
...
swh/deposit/tests/cli/test_client.py::test_cli_single_minimal_deposit_with_slug
  /build/swh-deposit-4z7JyE/swh-deposit-0.10.1/swh/deposit/cli/client.py:456: DeprecationWarning: "--slug" flag has been deprecated in favor of "--create-origin" flag. Please, start using "--create-origin" instead of "--slug"
    DeprecationWarning,
(^ repeated a gazillion time)

swh/deposit/tests/cli/test_client.py::test_cli_validation_metadata
  /build/swh-deposit-4z7JyE/swh-deposit-0.10.1/swh/deposit/cli/client.py:456: DeprecationWarning: "--slug" flag has been deprecated in favor of "--create-origin" flag. Please, start using "--create-origin" instead of "--slug"
    DeprecationWarning,
(^ repeated half a gazillion time)

swh/deposit/tests/loader/test_tasks.py::test_task_check_3
  /build/swh-deposit-4z7JyE/swh-deposit-0.10.1/swh/deposit/client.py:98: DeprecationWarning: "config" argument is deprecated, please use "url" and "auth" arguments instead; note that "auth" expects now a couple (username, password) and not a dict.
    DeprecationWarning,
(^ repeated a 16th of a gazillion time)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================================================================================================== 243 passed, 204 warnings in 39.78 seconds ==================================================================================================

whether mypy is happy is another matter though...

whether mypy is happy is another matter though...

well, it does not like it heh.

After rmultiple tryouts, settled on the last one which is more conventional.

1st which works (tests + type) was moving the annotation after the successful import.

try:
    from django.urls import URLPattern, URLResolver

    # Add types to urlpatterns if we can
    urlpatterns: Sequence[Union[URLPattern, URLResolver]]
except ImportError:
    # retro-compatibility import fix, django 1.11.29 [1] does not expose the previous
    # module, so we fallback to no typing for such version.
    # [1] django debian stable version: 1:1.11.29-1~deb10u1
    pass

But @vlorentz made me realize this is the intent of the future annotation, so:

from __future__ import annotations

try:
    from django.urls import URLPattern, URLResolver
except ImportError:
    # retro-compatibility import fix, django 1.11.29 [1] does not expose the previous
    # module, so we fallback to no typing for such version.
    # [1] django debian stable version: 1:1.11.29-1~deb10u1
    pass