Page MenuHomeSoftware Heritage
Differential D4826

Allow to use the JAVA_HOME environment for cassandra tests
ClosedPublic
Actions

Authored by douardda on Jan 8 2021, 2:10 PM.

Details

Reviewers
vlorentz
Group Reviewers
Reviewers
Commits
rDSTO728c3eeaae84: Allow to use the JAVA_HOME environment for cassandra tests
Summary

This allows to enforce a specific version of java to be used. For
example, since cassandra seems not to support java 14 yet, this allows
to run tests on bullseye:

JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ pytest swh

Diff Detail

Event Timeline

douardda created this revision.Jan 8 2021, 2:10 PM
Herald added a reviewer: Reviewers. · View Herald TranscriptJan 8 2021, 2:10 PM
swh-public-ci added a comment.Jan 8 2021, 2:20 PM

Build has FAILED

Patch application report for D4826 (id=17085)

Could not rebase; Attempt merge onto 2b35198d30...

Updating 2b35198d..fde2589c
Fast-forward
 swh/storage/cassandra/schema.py     | 139 +++++++++++++++++-------------------
 swh/storage/tests/test_cassandra.py |  14 ++--
 2 files changed, 75 insertions(+), 78 deletions(-)
Changes applied before test
commit fde2589cbf3295dd1b41423f65cb07ef269e7964
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:39:42 2021 +0100

    Allow to use the JAVA_HOME environment for cassandra tests
    
    This allows to enforce a specific version of java to be used. For
    example, since cassandra seems not to support java 14 yet, this allows
    to run tests on bullseye:
    
      JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ pytest swh

commit 44efbe3cae717d2de3acea20fa7111d0314d8835
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:44:25 2021 +0100

    Make the CREATE_TABLES_QUERIES in cassandra/schema.py an explicit list
    
    prevent being fooled by a missing '\n'.

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1082/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1082/console

Harbormaster returned this revision to the author for changes because remote builds failed.Jan 8 2021, 2:20 PM
Harbormaster failed remote builds in B18225: Diff 17085!
douardda updated this revision to Diff 17087.Jan 8 2021, 2:28 PM

Rebase

Depends on D4825

douardda added a parent revision: D4825: Make the CREATE_TABLES_QUERIES in cassandra/schema.py an explicit list.Jan 8 2021, 2:28 PM
douardda updated this revision to Diff 17088.Jan 8 2021, 2:31 PM

also adapt the tox.ini file

swh-public-ci added a comment.Jan 8 2021, 2:33 PM

Build is green

Patch application report for D4826 (id=17087)

Could not rebase; Attempt merge onto 2b35198d30...

Updating 2b35198d..b097634f
Fast-forward
 swh/storage/cassandra/schema.py     | 136 +++++++++++++++++-------------------
 swh/storage/tests/test_cassandra.py |  16 +++--
 2 files changed, 73 insertions(+), 79 deletions(-)
Changes applied before test
commit b097634fe1a9b4ec5d6ad180a1b3443488ccded3
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:39:42 2021 +0100

    Allow to use the JAVA_HOME environment for cassandra tests
    
    This allows to enforce a specific version of java to be used. For
    example, since cassandra seems not to support java 14 yet, this allows
    to run tests on bullseye:
    
      JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ pytest swh

commit 74e6f58e2f9b3e03b911dbe95b1f53f85db8fd42
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:44:25 2021 +0100

    Make the CREATE_TABLES_QUERIES in cassandra/schema.py an explicit list
    
    prevent being fooled by a missing '\n'.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1084/ for more details.

Harbormaster completed remote builds in B18227: Diff 17087.Jan 8 2021, 2:33 PM
swh-public-ci added a comment.Jan 8 2021, 2:37 PM

Build is green

Patch application report for D4826 (id=17088)

Could not rebase; Attempt merge onto 2b35198d30...

Updating 2b35198d..d9ddec59
Fast-forward
 swh/storage/cassandra/schema.py     | 136 +++++++++++++++++-------------------
 swh/storage/tests/test_cassandra.py |  16 +++--
 tox.ini                             |   1 +
 3 files changed, 74 insertions(+), 79 deletions(-)
Changes applied before test
commit d9ddec59d5ac5c36a2f32d495ea89046d3db621e
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:39:42 2021 +0100

    Allow to use the JAVA_HOME environment for cassandra tests
    
    This allows to enforce a specific version of java to be used. For
    example, since cassandra seems not to support java 14 yet, this allows
    to run tests on bullseye:
    
      JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ pytest swh

commit 74e6f58e2f9b3e03b911dbe95b1f53f85db8fd42
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:44:25 2021 +0100

    Make the CREATE_TABLES_QUERIES in cassandra/schema.py an explicit list
    
    prevent being fooled by a missing '\n'.

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1085/ for more details.

Harbormaster completed remote builds in B18228: Diff 17088.Jan 8 2021, 2:38 PM
douardda requested review of this revision.Jan 8 2021, 2:38 PM
vlorentz added a subscriber: vlorentz.Jan 8 2021, 2:38 PM

could you mention it in the README?

vlorentz accepted this revision.Jan 8 2021, 2:38 PM
This revision is now accepted and ready to land.Jan 8 2021, 2:38 PM
douardda updated this revision to Diff 17164.Jan 13 2021, 9:45 AM

Add a paragraph in the README file

swh-public-ci added a comment.Jan 13 2021, 9:50 AM

Build has FAILED

Patch application report for D4826 (id=17164)

Rebasing onto 74e6f58e2f...

Current branch diff-target is up to date.
Changes applied before test
commit 3df990dcefa45932f1257a5f31be1d83d26af822
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:39:42 2021 +0100

    Allow to use the JAVA_HOME environment for cassandra tests
    
    This allows to enforce a specific version of java to be used. For
    example, since cassandra seems not to support java 14 yet, this allows
    to run tests on bullseye:
    
      JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ pytest swh

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1093/
See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1093/console

Harbormaster failed remote builds in B18308: Diff 17164!Jan 13 2021, 9:50 AM
douardda updated this revision to Diff 17168.Jan 13 2021, 10:57 AM

rebase

swh-public-ci added a comment.Jan 13 2021, 11:02 AM

Build is green

Patch application report for D4826 (id=17168)

Rebasing onto 30945a5890...

Current branch diff-target is up to date.
Changes applied before test
commit 728c3eeaae84a7e249a712a5545cc9ba682403c3
Author: David Douard <david.douard@sdfa3.org>
Date:   Fri Jan 8 10:39:42 2021 +0100

    Allow to use the JAVA_HOME environment for cassandra tests
    
    This allows to enforce a specific version of java to be used. For
    example, since cassandra seems not to support java 14 yet, this allows
    to run tests on bullseye:
    
      JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ pytest swh

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/1095/ for more details.

Harbormaster completed remote builds in B18312: Diff 17168.Jan 13 2021, 11:02 AM
Closed by commit rDSTO728c3eeaae84: Allow to use the JAVA_HOME environment for cassandra tests (authored by douardda). · Explain WhyJan 13 2021, 11:05 AM
This revision was automatically updated to reflect the committed changes.
douardda added a commit: rDSTO728c3eeaae84: Allow to use the JAVA_HOME environment for cassandra tests.

Revision Contents
Changeset List

PathSizeCoverage (All)Coverage (Touched)
11 lines--
swh/
storage/
tests/
16 lines89%
Loading...
1 line--

Diff 17168

View Options

README.md

Show All 17 Lines
if 'GZip' in MIDDLEWARE[0]: if 'GZip' in MIDDLEWARE[0]:
MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware') MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware')
else: else:
MIDDLEWARE.insert(0, 'django.middleware.cache.UpdateCacheMiddleware') MIDDLEWARE.insert(0, 'django.middleware.cache.UpdateCacheMiddleware')
MIDDLEWARE += ['swh.web.common.middlewares.HtmlMinifyMiddleware', MIDDLEWARE += ['swh.web.common.middlewares.HtmlMinifyMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware'] 'django.middleware.cache.FetchFromCacheMiddleware']
if swh_web_config['throttling']['cache_uri']:
ardumontUnsubmitted
Not Done
Inline Actions

Shouldn't this be less strict (using dict's get method instead)?

ardumont: Shouldn't this be less strict (using dict's get method instead)?
CACHES.update({ CACHES.update({
'default': { 'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': swh_web_config['throttling']['cache_uri'], 'LOCATION': swh_web_config['throttling']['cache_uri'],
} }
}) })
# Setup support for proxy headers # Setup support for proxy headers
USE_X_FORWARDED_HOST = True USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# We're going through seven (or, in that case, 2) proxies thanks to Varnish # We're going through seven (or, in that case, 2) proxies thanks to Varnish
REST_FRAMEWORK['NUM_PROXIES'] = 2 REST_FRAMEWORK['NUM_PROXIES'] = 2
Show All 14 Lines
View Options

swh/storage/tests/test_cassandra.py

Show All 17 Lines
if 'GZip' in MIDDLEWARE[0]: if 'GZip' in MIDDLEWARE[0]:
MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware') MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware')
else: else:
MIDDLEWARE.insert(0, 'django.middleware.cache.UpdateCacheMiddleware') MIDDLEWARE.insert(0, 'django.middleware.cache.UpdateCacheMiddleware')
MIDDLEWARE += ['swh.web.common.middlewares.HtmlMinifyMiddleware', MIDDLEWARE += ['swh.web.common.middlewares.HtmlMinifyMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware'] 'django.middleware.cache.FetchFromCacheMiddleware']
if swh_web_config['throttling']['cache_uri']:
ardumontUnsubmitted
Not Done
Inline Actions

Shouldn't this be less strict (using dict's get method instead)?

ardumont: Shouldn't this be less strict (using dict's get method instead)?
CACHES.update({ CACHES.update({
'default': { 'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': swh_web_config['throttling']['cache_uri'], 'LOCATION': swh_web_config['throttling']['cache_uri'],
} }
}) })
# Setup support for proxy headers # Setup support for proxy headers
USE_X_FORWARDED_HOST = True USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# We're going through seven (or, in that case, 2) proxies thanks to Varnish # We're going through seven (or, in that case, 2) proxies thanks to Varnish
REST_FRAMEWORK['NUM_PROXIES'] = 2 REST_FRAMEWORK['NUM_PROXIES'] = 2
Show All 14 Lines
View Options

tox.ini

Show All 17 Lines
if 'GZip' in MIDDLEWARE[0]: if 'GZip' in MIDDLEWARE[0]:
MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware') MIDDLEWARE.insert(1, 'django.middleware.cache.UpdateCacheMiddleware')
else: else:
MIDDLEWARE.insert(0, 'django.middleware.cache.UpdateCacheMiddleware') MIDDLEWARE.insert(0, 'django.middleware.cache.UpdateCacheMiddleware')
MIDDLEWARE += ['swh.web.common.middlewares.HtmlMinifyMiddleware', MIDDLEWARE += ['swh.web.common.middlewares.HtmlMinifyMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware'] 'django.middleware.cache.FetchFromCacheMiddleware']
if swh_web_config['throttling']['cache_uri']:
ardumontUnsubmitted
Not Done
Inline Actions

Shouldn't this be less strict (using dict's get method instead)?

ardumont: Shouldn't this be less strict (using dict's get method instead)?
CACHES.update({ CACHES.update({
'default': { 'default': {
'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
'LOCATION': swh_web_config['throttling']['cache_uri'], 'LOCATION': swh_web_config['throttling']['cache_uri'],
} }
}) })
# Setup support for proxy headers # Setup support for proxy headers
USE_X_FORWARDED_HOST = True USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# We're going through seven (or, in that case, 2) proxies thanks to Varnish # We're going through seven (or, in that case, 2) proxies thanks to Varnish
REST_FRAMEWORK['NUM_PROXIES'] = 2 REST_FRAMEWORK['NUM_PROXIES'] = 2
Show All 14 Lines
About · Developer information · Legal