Page MenuHomeSoftware Heritage

Replace Sqlite with Postgres in unit tests
ClosedPublic

Authored by jayeshv on Sep 29 2021, 1:46 PM.

Details

Summary

Use PostgreSQL backend for django database in tests (T3601)

  • Added reset_sequence=True in the db fixture
  • Fixed timestamp comparison issue by ignoring microsecond

Diff Detail

Repository
rDWAPPS Web applications
Branch
T3601
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24084
Build 37570: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 37569: arc lint + arc unit

Unit TestsFailed

TimeTest
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_accept_pending_save_request
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
2 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_add_authorized_origin_url
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_add_unauthorized_origin_url
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_reject_pending_save_request
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_remove_authorized_origin_url
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
View Full Test Results (56 Failed · 496 Passed · 5 Skipped)

Event Timeline

Build has FAILED

Patch application report for D6372 (id=23168)

Rebasing onto 36f7ed0e5d...

Current branch diff-target is up to date.
Changes applied before test
commit da9de3d54ace58075849fed9c2c6ea2df6dd966b
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Sep 29 13:41:04 2021 +0200

    Replace Sqlite with Postgres in unit tests
    
    Use PostgreSQL backend for django database in tests (T3601)
    - Added reset_sequence=True in the db fixture
    - Fixed timestamp comparison issue by ignoring microsecond

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1127/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1127/console

Harbormaster returned this revision to the author for changes because remote builds failed.Sep 29 2021, 1:52 PM
Harbormaster failed remote builds in B24084: Diff 23168!

Replace Sqlite with Postgres in unit tests

Use PostgreSQL backend for django database in tests (T3601)

  • Added django_db_setup fixture
  • Added reset_sequence=True in the db fixture
  • Fixed timestamp comparison issue by ignoring microseconds
  • Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)

Build has FAILED

Patch application report for D6372 (id=23210)

Rebasing onto 7843408037...

Current branch diff-target is up to date.
Changes applied before test
commit 15dca7996be3f589cdd0a7ce0c518360ab882273
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Thu Sep 30 13:28:56 2021 +0200

    Replace Sqlite with Postgres in unit tests
    
    Use PostgreSQL backend for django database in tests (T3601)
    - Added reset_sequence=True in the db fixture
    - Fixed timestamp comparison issue by ignoring microseconds
    - Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1129/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1129/console

Harbormaster returned this revision to the author for changes because remote builds failed.Sep 30 2021, 1:40 PM
Harbormaster failed remote builds in B24127: Diff 23210!

Replace Sqlite with Postgres in unit tests

Use PostgreSQL backend in django database in tests

  • Added reset_sequence=True in the db fixture
  • Fixed timestamp comparison issue by ignoring microseconds
  • Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)

Changed conftest to use sqlite3 for frontend tests

related to T3601

Build is green

Patch application report for D6372 (id=23221)

Rebasing onto 7843408037...

Current branch diff-target is up to date.
Changes applied before test
commit 5ec4a98ff749fe73a0455460e9894a0542c1ab23
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Fri Oct 1 11:41:16 2021 +0200

    Replace Sqlite with Postgres in unit tests
    
     Use PostgreSQL backend for django database in tests (T3601)
    - Added reset_sequence=True in the db fixture
    - Fixed timestamp comparison issue by ignoring microseconds
    - Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)
    Changed conftest to use sqlite3 for frontend tests

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1130/ for more details.

anlambert added a subscriber: anlambert.

Looks good to me, I added two nitpick comments for small improvements.

requirements.txt
25 ↗(On Diff #23221)

You should rather pin to < 2.9 in case there is a new patch release for the 2.8 series.

swh/web/tests/auth/test_views.py
172–174

Could you add a comment explaining why dropping milliseconds for the test to succeed ?

This revision is now accepted and ready to land.Oct 1 2021, 12:35 PM

Fixed the issue in timestamp comparison

Build is green

Patch application report for D6372 (id=23231)

Rebasing onto 7843408037...

Current branch diff-target is up to date.
Changes applied before test
commit 224356feace82f6c20dfbcd518b8ba02b1f3bd54
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Fri Oct 1 13:54:19 2021 +0200

    Use PostgreSQL backend in django database in tests
    
    Added reset_sequence=True in the db fixture
    Pinned psycopg2 to < 2.9 (To fix UTC related assertion error)
    Changed conftest to use sqlite3 for frontend tests

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1131/ for more details.

Fixed the issue in timestamp comparison

How did you manage to fix it ? I got the microseconds comparison issue that morning when testing the changes.

Build is green

Patch application report for D6372 (id=23232)

Rebasing onto 7843408037...

Current branch diff-target is up to date.
Changes applied before test
commit a41f090641760119f33fb4beb2187727df7c2d0c
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Fri Oct 1 14:00:22 2021 +0200

    Use PostgreSQL backend in django database in tests
    
    Added reset_sequence=True in the db fixture
    Pinned psycopg2 to < 2.9 (To fix UTC related assertion error)
    Changed conftest to use sqlite3 for frontend tests

See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1132/ for more details.