Page MenuHomeSoftware Heritage
Differential D4384

debian/*: Separate python3-swh.core.db.pytestplugin package from python3-swh.core
ClosedPublic
Actions

Authored by ardumont on Oct 29 2020, 4:24 PM.

Details

Reviewers
olasd
Group Reviewers
System administrators
Maniphest Tasks
T2746: packaging: Split swh.core and test dependencies properly
Summary

This will allow to define more properly the dependencies between our debian
packages. Typically some db modules (storage, scheduler, vault, ...) should
require this module as build/test dependencies instead of the plain
python3-swh.core.

Related to D4376#108929
Related to T2746

Test Plan

gbp:

...
python3-swh.core.db.pytestplugin_0.9.0-1~swh2_all.deb
-----------------------------------------------------

 new Debian package, version 2.0.
 size 12228 bytes: control archive=1072 bytes.
     719 bytes,    13 lines      control
     285 bytes,     3 lines      md5sums
     289 bytes,    12 lines   *  postinst             #!/bin/sh
     438 bytes,    12 lines   *  prerm                #!/bin/sh
 Package: python3-swh.core.db.pytestplugin
 Source: swh-core
 Version: 0.9.0-1~swh2
 Architecture: all
 Maintainer: Software Heritage developers <swh-devel@inria.fr>
 Installed-Size: 31
 Depends: python3-pytest-postgresql, python3-swh.core (= 0.9.0-1~swh2), python3-aiohttp, python3-aiohttp-utils, python3-arrow, python3-blinker, python3-decorator, python3-flask, python3-iso8601, python3-msgpack (>= 0.5~), python3-psycopg2, python3-requests, python3-typing-extensions, python3:any
 Breaks: python3-swh.core (<< 0.9.0-1~swh2~)
 Replaces: python3-swh.core (<< 0.9.0-1~swh2~)
 Section: python
 Priority: optional
 Homepage: https://forge.softwareheritage.org/diffusion/DCORE/
 Description: Software Heritage Core Db Pytest Utilities

drwxr-xr-x root/root         0 2020-11-20 16:37 ./
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/db/
-rw-r--r-- root/root      5903 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/pytest_plugin.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/share/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/share/doc/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/share/doc/python3-swh.core.db.pytestplugin/
-rw-r--r-- root/root      7453 2020-11-20 16:37 ./usr/share/doc/python3-swh.core.db.pytestplugin/changelog.Debian.gz
-rw-r--r-- root/root      2103 2020-11-20 16:37 ./usr/share/doc/python3-swh.core.db.pytestplugin/copyright


python3-swh.core_0.9.0-1~swh2_all.deb
-------------------------------------

 new Debian package, version 2.0.
 size 1119060 bytes: control archive=3276 bytes.
     629 bytes,    11 lines      control
    8357 bytes,    85 lines      md5sums
     257 bytes,    12 lines   *  postinst             #!/bin/sh
     406 bytes,    12 lines   *  prerm                #!/bin/sh
 Package: python3-swh.core
 Source: swh-core
 Version: 0.9.0-1~swh2
 Architecture: all
 Maintainer: Software Heritage developers <swh-devel@inria.fr>
 Installed-Size: 1445
 Depends: python3-aiohttp, python3-aiohttp-utils, python3-arrow, python3-blinker, python3-click, python3-decorator, python3-deprecated, python3-flask, python3-iso8601, python3-msgpack (>= 0.5~), python3-psycopg2, python3-requests, python3-sentry-sdk, python3-typing-extensions, python3-yaml, python3:any, python3-systemd
 Section: python
 Priority: optional
 Homepage: https://forge.softwareheritage.org/diffusion/DCORE/
 Description: Software Heritage core utilities

drwxr-xr-x root/root         0 2020-11-20 16:37 ./
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/bin/
-rwxr-xr-x root/root       953 2020-11-20 16:37 ./usr/bin/swh
-rwxr-xr-x root/root       969 2020-11-20 16:37 ./usr/bin/swh-db-init
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh.core-0.9.0.egg-info/
-rw-r--r-- root/root      4121 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh.core-0.9.0.egg-info/PKG-INFO
-rw-r--r-- root/root         1 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh.core-0.9.0.egg-info/dependency_links.txt
-rw-r--r-- root/root       230 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh.core-0.9.0.egg-info/entry_points.txt
-rw-r--r-- root/root       498 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh.core-0.9.0.egg-info/requires.txt
-rw-r--r-- root/root        20 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh.core-0.9.0.egg-info/top_level.txt
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/
-rw-r--r-- root/root       127 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/__init__.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/
-rw-r--r-- root/root         0 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/__init__.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/api/
-rw-r--r-- root/root     16061 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/__init__.py
-rw-r--r-- root/root      6328 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/asynchronous.py
-rw-r--r-- root/root      1205 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/classes.py
-rw-r--r-- root/root      1093 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/gunicorn_config.py
-rw-r--r-- root/root      5818 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/negotiation.py
-rw-r--r-- root/root      9748 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/serializers.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/api/tests/
-rw-r--r-- root/root         0 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/__init__.py
-rw-r--r-- root/root         0 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/conftest.py
-rw-r--r-- root/root      4290 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/server_testing.py
-rw-r--r-- root/root      7863 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_async.py
-rw-r--r-- root/root      1905 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_classes.py
-rw-r--r-- root/root      3779 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_gunicorn.py
-rw-r--r-- root/root       862 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_init.py
-rw-r--r-- root/root      2667 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_rpc_client.py
-rw-r--r-- root/root      4024 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_rpc_client_server.py
-rw-r--r-- root/root      4396 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_rpc_server.py
-rw-r--r-- root/root      4871 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_rpc_server_asynchronous.py
-rw-r--r-- root/root      8318 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api/tests/test_serializers.py
-rw-r--r-- root/root        63 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/api_async.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/cli/
-rw-r--r-- root/root      4300 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/cli/__init__.py
-rw-r--r-- root/root     10387 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/cli/db.py
-rw-r--r-- root/root      2050 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/collections.py
-rw-r--r-- root/root      8511 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/config.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/db/
-rw-r--r-- root/root     10619 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/__init__.py
-rw-r--r-- root/root      3524 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/common.py
-rw-r--r-- root/root      8357 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/db_utils.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/db/tests/
-rw-r--r-- root/root         0 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/__init__.py
-rw-r--r-- root/root        44 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/conftest.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/db/tests/data/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/db/tests/data/cli/
-rw-r--r-- root/root        41 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/data/cli/0-superuser-init.sql
-rw-r--r-- root/root       307 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/data/cli/1-schema.sql
-rw-r--r-- root/root       156 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/data/cli/3-func.sql
-rw-r--r-- root/root       245 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/data/cli/4-data.sql
-rw-r--r-- root/root      8801 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/db_testing.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/db/tests/pytest_plugin/
-rw-r--r-- root/root         0 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/pytest_plugin/__init__.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/db/tests/pytest_plugin/data/
-rw-r--r-- root/root       425 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/pytest_plugin/data/0-schema.sql
-rw-r--r-- root/root       940 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/pytest_plugin/data/1-data.sql
-rw-r--r-- root/root      4442 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/pytest_plugin/test_pytest_plugin.py
-rw-r--r-- root/root      8949 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/test_cli.py
-rw-r--r-- root/root     12906 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/db/tests/test_db.py
-rw-r--r-- root/root      4331 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/logger.py
-rw-r--r-- root/root        27 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/py.typed
-rw-r--r-- root/root     10299 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/pytest_plugin.py
-rw-r--r-- root/root      1094 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/sentry.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/sql/
-rw-r--r-- root/root      1013 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/sql/log-schema.sql
-rw-r--r-- root/root     14349 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/statsd.py
-rw-r--r-- root/root      4339 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tarball.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/
-rw-r--r-- root/root        98 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/__init__.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/
-rw-r--r-- root/root   1087901 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/groff-1.02.tar.Z
-rw-r--r-- root/root     10240 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/hello.tar
-rw-r--r-- root/root       199 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/hello.tar.bz2
-rw-r--r-- root/root       181 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/hello.tar.gz
-rw-r--r-- root/root       190 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/hello.tar.lz
-rw-r--r-- root/root     10240 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/hello.tar.x
-rw-r--r-- root/root       162 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/archives/hello.zip
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/http_example.com/
-rw-r--r-- root/root        12 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/http_example.com/something.json
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_example.com/
-rw-r--r-- root/root        23 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_example.com/file.json
-rw-r--r-- root/root        28 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_example.com/file.json,name=doe,firstname=jane
-rw-r--r-- root/root        25 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_example.com/file.json_visit1
-rw-r--r-- root/root         9 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_example.com/other.json
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_forge.s.o/
-rw-r--r-- root/root        12 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_forge.s.o/api_diffusion,attachments[uris]=1
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_www.reference.com/
-rw-r--r-- root/root        17 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/data/https_www.reference.com/web,q=What+Is+an+Example+of+a+URL?,qo=contentPageRelatedSearch,o=600605,l=dir,sga=1
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/fixture/
-rw-r--r-- root/root         0 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/fixture/__init__.py
-rw-r--r-- root/root       401 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/fixture/conftest.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/fixture/data/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/lib/python3/dist-packages/swh/core/tests/fixture/data/https_example.com/
-rw-r--r-- root/root        25 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/fixture/data/https_example.com/file.json
-rw-r--r-- root/root       803 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/fixture/test_pytest_plugin.py
-rw-r--r-- root/root      8428 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_cli.py
-rw-r--r-- root/root      2439 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_collections.py
-rw-r--r-- root/root      9845 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_config.py
-rw-r--r-- root/root      3581 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_logger.py
-rw-r--r-- root/root      4010 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_pytest_plugin.py
-rw-r--r-- root/root     18705 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_statsd.py
-rw-r--r-- root/root      6518 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_tarball.py
-rw-r--r-- root/root      4076 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/tests/test_utils.py
-rw-r--r-- root/root      3544 2020-11-20 14:36 ./usr/lib/python3/dist-packages/swh/core/utils.py
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/share/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/share/doc/
drwxr-xr-x root/root         0 2020-11-20 16:37 ./usr/share/doc/python3-swh.core/
-rw-r--r-- root/root      7453 2020-11-20 16:37 ./usr/share/doc/python3-swh.core/changelog.Debian.gz
-rw-r--r-- root/root      2103 2020-11-20 16:37 ./usr/share/doc/python3-swh.core/copyright


+------------------------------------------------------------------------------+
| Post Build                                                                   |
+------------------------------------------------------------------------------+


+------------------------------------------------------------------------------+
| Cleanup                                                                      |
+------------------------------------------------------------------------------+

Purging /<<BUILDDIR>>
Not cleaning session: cloned chroot in use

+------------------------------------------------------------------------------+
| Summary                                                                      |
+------------------------------------------------------------------------------+

Build Architecture: amd64
Build Type: full
Build-Space: 10464
Build-Time: 25
Distribution: unstable-swh
Host Architecture: amd64
Install-Time: 77
Job: /home/tony/debian/build-area/swh-core_0.9.0-1~swh2.dsc
Machine Architecture: amd64
Package: swh-core
Package-Time: 152
Source-Version: 0.9.0-1~swh2
Space: 10464
Status: successful
Version: 0.9.0-1~swh2
--------------------------------------------------------------------------------
Finished at 2020-11-20T16:43:21Z
Build needed 00:02:32, 10464k disk space

Diff Detail

Repository
rDCORE Foundations and core functionalities
Branch
debian/unstable-swh
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 17304
Build 26725: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 26724: arc lint + arc unit

Event Timeline

ardumont created this revision.Oct 29 2020, 4:24 PM
swh-public-ci added a comment.Oct 29 2020, 4:24 PM
This comment was removed by ardumont.
Harbormaster failed remote builds in B16727: Diff 15501!Oct 29 2020, 4:24 PM
ardumont added a comment.Oct 29 2020, 4:26 PM

This comment was removed by ardumont.

merge commit which somehow failed.
It did not bring anything except a gazillion lines in the diff...

ardumont added a comment.Oct 29 2020, 4:28 PM

The requirements are a bit off in the output but that's because i need to rework some requirements-*.txt in the master as well... (i think).

ardumont retitled this revision from debian/*: Create python3-swh.core.db package to debian/*: Separate python3-swh.core.db package from python3-swh.core.Oct 29 2020, 4:36 PM

unsure yet if that supersedes completely D4376 or not though.

ardumont edited the summary of this revision. (Show Details)Oct 30 2020, 11:02 AM
ardumont edited the summary of this revision. (Show Details)Oct 30 2020, 1:57 PM
ardumont planned changes to this revision.Oct 30 2020, 3:01 PM

I misunderstood the hint.
It's not about separating core.db from core, it's to create a dedicated python3-swh.core.db.pytest-plugin (<- or something).

i'll check later.

ardumont edited the summary of this revision. (Show Details)Nov 2 2020, 10:59 AM
ardumont added a task: T2746: packaging: Split swh.core and test dependencies properly.
ardumont updated this revision to Diff 15618.Nov 3 2020, 5:59 PM

Actually create a python3-swh.core.db.pytest package

swh-debian-build-unstable-swh seems happy but the content seems off...
(i need to further investigate...)

swh-public-ci added a comment.Nov 3 2020, 6:00 PM
This comment was removed by ardumont.
Harbormaster failed remote builds in B16841: Diff 15618!Nov 3 2020, 6:00 PM
ardumont updated this revision to Diff 15619.Nov 3 2020, 6:01 PM

Fix mv -v is not supported somehow

swh-public-ci added a comment.Nov 3 2020, 6:01 PM
This comment was removed by ardumont.
Harbormaster failed remote builds in B16842: Diff 15619!Nov 3 2020, 6:01 PM
ardumont planned changes to this revision.Nov 3 2020, 6:02 PM
ardumont added a comment.EditedNov 3 2020, 6:04 PM

gbp build-package... seems happy but the content seems off...

python3-swh.core.db.pytest_0.8.0-1~swh1_all.deb
-----------------------------------------------

 new Debian package, version 2.0.
 size 9548 bytes: control archive=660 bytes.
     575 bytes,    11 lines      control
     180 bytes,     2 lines      md5sums
 Package: python3-swh.core.db.pytest
 Source: swh-core
 Version: 0.8.0-1~swh1
 Architecture: all
 Maintainer: Software Heritage developers <swh-devel@inria.fr>
 Installed-Size: 17
 Depends: python3-pytest-postgresql, python3-psycopg2, python3-typing-extensions, python3-aiohttp, python3-aiohttp-utils, python3-arrow, python3-blinker, python3-decorator, python3-flask, python3-iso8601, python3-msgpack (>= 0.5~), python3-requests
 Section: python
 Priority: optional
 Homepage: https://forge.softwareheritage.org/diffusion/DCORE/
 Description: Software Heritage core db pytest utilities

drwxr-xr-x root/root         0 2020-10-30 08:13 ./
drwxr-xr-x root/root         0 2020-10-30 08:13 ./usr/
drwxr-xr-x root/root         0 2020-10-30 08:13 ./usr/share/
drwxr-xr-x root/root         0 2020-10-30 08:13 ./usr/share/doc/
drwxr-xr-x root/root         0 2020-10-30 08:13 ./usr/share/doc/python3-swh.core.db.pytest/
-rw-r--r-- root/root      7196 2020-10-30 08:13 ./usr/share/doc/python3-swh.core.db.pytest/changelog.Debian.gz
-rw-r--r-- root/root      2103 2020-10-30 08:13 ./usr/share/doc/python3-swh.core.db.pytest/copyright

...

¯\_(ツ)_/¯...

ardumont added inline comments.Nov 20 2020, 10:47 AM
debian/control
55

well, you named .pytest here but you used pytest_plugin path in the rules file...

ardumont updated this revision to Diff 16106.Nov 20 2020, 11:16 AM

Make the package work

  • python3-swh.core.db.pytestplugin with the right content and dependency
  • python3-swh.core without the pytest-postgresql dependency

My issue was a discrepancy in the package name between the debian/control and
debian/rules files.

Note: So it works (i'll update the description with some relevant build output.
Although, now i'm bypassing the {python3:Depends} keyword (so i removed the
dh_python3_override. I think i need to rework the python3 declaration on master
instead.

@olasd ^

swh-public-ci added a comment.Nov 20 2020, 11:16 AM
This comment was removed by ardumont.
Harbormaster failed remote builds in B17304: Diff 16106!Nov 20 2020, 11:16 AM
ardumont edited the test plan for this revision. (Show Details)Nov 20 2020, 11:17 AM
ardumont added inline comments.Nov 20 2020, 11:22 AM
debian/control
53

I think if I want to avoid that, I need to do as my diff description says, rework the master branch dependencies somehow.

debian/rules
10

I removed it as it's no longer needed since in that vresion of the diff, i bypass the python3:Depends keywork (and I think those come together).

So, same remark "I think if I want to avoid that, I need to do as my diff description says, rework the master branch dependencies somehow."

ardumont added inline comments.Nov 20 2020, 11:23 AM
debian/control
53

"avoid that" as in avoid "inlining" the dependencies which are already declares somewhere else...

ardumont added inline comments.Nov 20 2020, 11:30 AM
debian/rules
19

The needs for those here is debatable...
I initially move those tests because they are using the fixture.

I think i'll change that:

  1. all tests are detected and when run pass during the build process
  2. I noticed there are other tests that are using the fixture i'm trying to separate in that dedicated package
  3. I was convinced that I added tests that covered specifically the fixture when I moved it in the core... so i'll get back to look for those.
ardumont added inline comments.Nov 20 2020, 11:37 AM
debian/rules
19

And, yes, i moved the wrong tests, the ones dedicated to test that fixture are swh/core/db/tests/test_db_utils.py...

olasd added inline comments.Nov 20 2020, 11:38 AM
debian/rules
10

I think we should:

  • drop pytest-postgresql from requirements-db.txt and move it to a separate requirements file
  • keep the original dh_python3 call
  • keep ${python3:Depends} on the main swh.core package
  • only have explicit dependencies on the separate pytest-plugin package.

I'm afraid maintaining this list of dependencies by hand will bite us sooner rather than later.

19

The tests run by the build process happen before dh_install, so the layout of the installed packages doesn't matter (for the build-time tests of swh.core).

My initial suggestion, which still stands, is to move only the pytest_plugin.py to another package, which other modules can build-depend on explicitly if they themselves have tests that require the swh.core.pytest_plugin.

That pytest-plugin-specific package would depend on only python3-swh.core and python3-pytest-postgresql (which in turn pulls the postgresql server package). All other dependencies should stay in the main python3-swh.core package.

ardumont added inline comments.Nov 20 2020, 11:53 AM
debian/rules
10

yes, agreed.

I was coming to that conclusion (see my comment just before yours).
Was unsure whether we opened yet another requirement-db-pytestplugin.txt (or something) for it.

So here this diff, for now, can only solve one part of the problem.
Going to solve the other part.

19

but ok, with your remark, i grok that's no longer a concern.
Just move swh.core.db.pytest_plugin!

I got sidetracked by my initial blocking point (the files were not moved).
Then given the time span of this diff, i forgot your initial comment and got carried away ¯\_(ツ)_/¯

Thanks.

ardumont updated this revision to Diff 16108.Nov 20 2020, 11:58 AM

Adapt according to review, fixes the first part of the separation problem:

  • keep the python3:Depends to avoid a manual hellish maintenance of the core package (my previous attempt was an iterative step)
  • python3-swh.core dependencies as before
  • python3-swh.core.db.pytestplugin runtime depends on python3-swh.core and python3-pytest-postgresql

Right now the build should still mention pytest-postgresql as a dependency for
python3-swh.core because some work is needed on the master branch of the
repository.

So another diff for the master branch should be coming soon to deal with that (the
requirement*.txt part to adapt).

swh-public-ci added a comment.Nov 20 2020, 11:58 AM
This comment was removed by ardumont.
Harbormaster failed remote builds in B17306: Diff 16108!Nov 20 2020, 11:58 AM
ardumont edited the test plan for this revision. (Show Details)Nov 20 2020, 12:02 PM
ardumont edited the test plan for this revision. (Show Details)
ardumont edited the summary of this revision. (Show Details)Nov 20 2020, 1:54 PM

So, same remark "I think if I want to avoid that, I need to do as my diff description says, rework the master branch dependencies somehow."

drop pytest-postgresql from requirements-db.txt and move it to a separate requirements file

So another diff for the master branch should be coming soon to deal with that (the
requirement*.txt part to adapt).

D4543 for the other part of the puzzle

\o/

ardumont added inline comments.Nov 20 2020, 2:05 PM
debian/rules
14

from @olasd, i'll drop that file or otherwise both package will conflict.

ardumont planned changes to this revision.Nov 20 2020, 2:06 PM
ardumont edited the test plan for this revision. (Show Details)Nov 20 2020, 2:21 PM
ardumont updated this revision to Diff 16116.Nov 20 2020, 2:22 PM
ardumont edited the test plan for this revision. (Show Details)

Drop the init from swh.core.db arborescence. It will conflict with the
python3-swh.core package and it will be there already as
python3-swh.core.db.pytestplugin depends on core.

swh-public-ci added a comment.Nov 20 2020, 2:23 PM
This comment was removed by ardumont.
Harbormaster failed remote builds in B17314: Diff 16116!Nov 20 2020, 2:23 PM
ardumont mentioned this in D4376: db.pytest_plugin: Make pytest-postgresql an optional runtime dependency.Nov 20 2020, 2:26 PM
ardumont edited the test plan for this revision. (Show Details)Nov 20 2020, 3:56 PM
ardumont updated this revision to Diff 16133.Nov 20 2020, 3:57 PM
ardumont edited the test plan for this revision. (Show Details)
  • Ensure it works with the now released adaptation on swh.core (which separates

swh.core.db.pytest_plugin requirements from the rest) -> it's ok now (updated
the test plan with the output)

  • Adds the Breaks and Replaces keyword
swh-public-ci added a comment.Nov 20 2020, 3:58 PM
This comment was removed by ardumont.
Harbormaster failed remote builds in B17333: Diff 16133!Nov 20 2020, 3:58 PM
ardumont edited the summary of this revision. (Show Details)Nov 20 2020, 4:00 PM
ardumont updated this revision to Diff 16141.Nov 20 2020, 5:02 PM
  • Move the Breaks/Replaces to the right package...
  • Add ~ to the version break/replace
swh-public-ci added a comment.Nov 20 2020, 5:02 PM
This comment was removed by ardumont.
Harbormaster failed remote builds in B17341: Diff 16141!Nov 20 2020, 5:02 PM
ardumont edited the test plan for this revision. (Show Details)Nov 20 2020, 5:03 PM
ardumont edited the test plan for this revision. (Show Details)Nov 20 2020, 5:43 PM
ardumont updated this revision to Diff 16152.Nov 20 2020, 5:45 PM
ardumont edited the test plan for this revision. (Show Details)
  • Revert my last changes (it was ok, i misunderstood what i needed to do ¯\_(ツ)_/¯)
  • Bump a new changelog entry 0.9.0-1~swh2
  • And reference that version as the break/replace version
swh-public-ci added a comment.Nov 20 2020, 5:46 PM
This comment was removed by ardumont.
Harbormaster failed remote builds in B17351: Diff 16152!Nov 20 2020, 5:46 PM
olasd accepted this revision.Nov 20 2020, 6:57 PM
This revision is now accepted and ready to land.Nov 20 2020, 6:57 PM
ardumont retitled this revision from debian/*: Separate python3-swh.core.db package from python3-swh.core to debian/*: Separate python3-swh.core.db.pytestplugin package from python3-swh.core.Nov 20 2020, 9:14 PM
ardumont closed this revision.Nov 23 2020, 10:28 AM

pushed.

Revision Contents
Changeset List

PathSize
debian/
27 lines
14 lines

Diff 16106

View Options

debian/control

Loading...
View Options

debian/rules

Loading...
About · Developer information · Legal