Page MenuHomeSoftware Heritage
Differential D6372

Replace Sqlite with Postgres in unit tests
ClosedPublic
Actions

Authored by jayeshv on Sep 29 2021, 1:46 PM.

Details

Reviewers
anlambert
Group Reviewers
Reviewers
Maniphest Tasks
T3601: Use PostgreSQL backend for django database in tests
Commits
rDWAPPSa41f09064176: Use PostgreSQL backend in django database in tests
Summary

Use PostgreSQL backend for django database in tests (T3601)

  • Added reset_sequence=True in the db fixture
  • Fixed timestamp comparison issue by ignoring microsecond

Diff Detail

Repository
rDWAPPS Web applications
Branch
T3601
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 24084
Build 37570: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 37569: arc lint + arc unit

Unit TestsFailed
View All

TimeTest
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_accept_pending_save_request
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
2 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_add_authorized_origin_url
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_add_unauthorized_origin_url
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_reject_pending_save_request
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
1 msJenkins > .tox.py3.lib.python3.7.site-packages.swh.web.tests.admin.test_origin_save::test_remove_authorized_origin_url
self = <django.db.backends.postgresql.base.DatabaseWrapper object at 0x7f869ea1c518> def ensure_connection(self):
View Full Test Results (56 Failed · 496 Passed · 5 Skipped)

Event Timeline

jayeshv created this revision.Sep 29 2021, 1:46 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptSep 29 2021, 1:46 PM
swh-public-ci added a comment.Sep 29 2021, 1:52 PM

Build has FAILED

Patch application report for D6372 (id=23168)

Rebasing onto 36f7ed0e5d...

Current branch diff-target is up to date.
Changes applied before test
commit da9de3d54ace58075849fed9c2c6ea2df6dd966b
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Wed Sep 29 13:41:04 2021 +0200

    Replace Sqlite with Postgres in unit tests
    
    Use PostgreSQL backend for django database in tests (T3601)
    - Added reset_sequence=True in the db fixture
    - Fixed timestamp comparison issue by ignoring microsecond

Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1127/
See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1127/console

Harbormaster returned this revision to the author for changes because remote builds failed.Sep 29 2021, 1:52 PM
Harbormaster failed remote builds in B24084: Diff 23168!
jayeshv updated this revision to Diff 23210.Sep 30 2021, 1:34 PM

Replace Sqlite with Postgres in unit tests




Use PostgreSQL backend for django database in tests (T3601)



    

  • Added django_db_setup fixture
    • 

  • Added reset_sequence=True in the db fixture
    • 

  • Fixed timestamp comparison issue by ignoring microseconds
    • 

  • Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)
    • 

swh-public-ci added a comment.Sep 30 2021, 1:40 PM
Build has FAILED



Patch application report for D6372 (id=23210)



Rebasing onto 7843408037...



Current branch diff-target is up to date.





Changes applied before test



commit 15dca7996be3f589cdd0a7ce0c518360ab882273
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Thu Sep 30 13:28:56 2021 +0200

    Replace Sqlite with Postgres in unit tests
    
    Use PostgreSQL backend for django database in tests (T3601)
    - Added reset_sequence=True in the db fixture
    - Fixed timestamp comparison issue by ignoring microseconds
    - Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)



Link to build: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1129/

See console output for more information: https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1129/console
Harbormaster returned this revision to the author for changes because remote builds failed.Sep 30 2021, 1:40 PM
Harbormaster failed remote builds in B24127: Diff 23210!
jayeshv updated this revision to Diff 23221.Oct 1 2021, 11:44 AM
Replace Sqlite with Postgres in unit tests






Use PostgreSQL backend in django database in tests



    

  • Added reset_sequence=True in the db fixture
    • 

  • Fixed timestamp comparison issue by ignoring microseconds
    • 

  • Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)
    • 




Changed conftest to use sqlite3 for frontend tests



related to T3601
swh-public-ci added a comment.Oct 1 2021, 11:58 AM
Build is green



Patch application report for D6372 (id=23221)



Rebasing onto 7843408037...



Current branch diff-target is up to date.





Changes applied before test



commit 5ec4a98ff749fe73a0455460e9894a0542c1ab23
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Fri Oct 1 11:41:16 2021 +0200

    Replace Sqlite with Postgres in unit tests
    
     Use PostgreSQL backend for django database in tests (T3601)
    - Added reset_sequence=True in the db fixture
    - Fixed timestamp comparison issue by ignoring microseconds
    - Pinned psycopg2 to 2.8.6 (To fix UTC related assertion error)
    Changed conftest to use sqlite3 for frontend tests



See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1130/ for more details.
Harbormaster completed remote builds in B24138: Diff 23221.Oct 1 2021, 11:58 AM
jayeshv requested review of this revision.Oct 1 2021, 11:58 AM
anlambert accepted this revision.Oct 1 2021, 12:35 PM
anlambert added a subscriber: anlambert.
Looks good to me, I added two nitpick comments for small improvements.
requirements.txt


25 ↗(On Diff #23221)
You should rather pin to < 2.9 in case there is a new patch release for the 2.8 series.


swh/web/tests/auth/test_views.py


172–174
Could you add a comment explaining why dropping milliseconds for the test to succeed ?
This revision is now accepted and ready to land.Oct 1 2021, 12:35 PM
jayeshv updated this revision to Diff 23231.Oct 1 2021, 1:56 PM
Fixed the issue in timestamp comparison
jayeshv updated this revision to Diff 23232.Oct 1 2021, 2:01 PM
Updated requirements
jayeshv marked 2 inline comments as done.Oct 1 2021, 2:01 PM
swh-public-ci added a comment.Oct 1 2021, 2:10 PM
Build is green



Patch application report for D6372 (id=23231)



Rebasing onto 7843408037...



Current branch diff-target is up to date.





Changes applied before test



commit 224356feace82f6c20dfbcd518b8ba02b1f3bd54
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Fri Oct 1 13:54:19 2021 +0200

    Use PostgreSQL backend in django database in tests
    
    Added reset_sequence=True in the db fixture
    Pinned psycopg2 to < 2.9 (To fix UTC related assertion error)
    Changed conftest to use sqlite3 for frontend tests



See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1131/ for more details.
Harbormaster completed remote builds in B24148: Diff 23231.Oct 1 2021, 2:10 PM
anlambert added a comment.Oct 1 2021, 2:14 PM


In D6372#165729, @jayeshv wrote:


Fixed the issue in timestamp comparison





How did you manage to fix it ? I got the microseconds comparison issue that morning when testing the changes.
swh-public-ci added a comment.Oct 1 2021, 2:17 PM
Build is green



Patch application report for D6372 (id=23232)



Rebasing onto 7843408037...



Current branch diff-target is up to date.





Changes applied before test



commit a41f090641760119f33fb4beb2187727df7c2d0c
Author: Jayesh Velayudhan <jayesh@softwareheritage.org>
Date:   Fri Oct 1 14:00:22 2021 +0200

    Use PostgreSQL backend in django database in tests
    
    Added reset_sequence=True in the db fixture
    Pinned psycopg2 to < 2.9 (To fix UTC related assertion error)
    Changed conftest to use sqlite3 for frontend tests



See https://jenkins.softwareheritage.org/job/DWAPPS/job/tests-on-diff/1132/ for more details.
Harbormaster completed remote builds in B24149: Diff 23232.Oct 1 2021, 2:17 PM
Closed by commit rDWAPPSa41f09064176: Use PostgreSQL backend in django database in tests (authored by jayeshv).  ·  Explain WhyOct 1 2021, 2:32 PM
This revision was automatically updated to reflect the committed changes.
jayeshv added a commit: rDWAPPSa41f09064176: Use PostgreSQL backend in django database in tests.
Revision Contents
Changeset List
PathSize
swh/
web/
2 lines
settings/
4 lines
tests/
auth/
18 lines
Diff 23168
View Options
swh/web/config.py
Loading...
View Options
swh/web/settings/tests.py
Loading...
View Options
swh/web/tests/auth/test_views.py
Loading...
About · Developer information · Legal