Page MenuHomeSoftware Heritage

tests: Allow cassandra binary path to be configured through env variable
ClosedPublic

Authored by ardumont on Jul 12 2020, 4:01 PM.

Details

Summary

The current hard-coded value won't work for other distributions not relying on
standard conventions (e.g. nixos...).

This keeps the original behavior and only allow to diverge based on the
environment variable BIN_CASSANDRA [1]
This also:

  • fixes an issue on log path inexistence which raises.
  • renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for consistency)

Note:
Currently trying to run this with my nixos box and this minimum prevents it.

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

ardumont created this revision.Jul 12 2020, 4:01 PM

Build is green

Patch application report for D3505 (id=12389)

Could not rebase; Attempt merge onto 9219a23606...

Updating 9219a236..67f7351c
Fast-forward
 README.md                           |  4 +++-
 sql/upgrades/158.sql                | 10 +++++-----
 swh/storage/sql/40-swh-func.sql     | 10 +++++-----
 swh/storage/tests/test_cassandra.py | 10 +++++++---
 tox.ini                             |  1 +
 5 files changed, 21 insertions(+), 14 deletions(-)
Changes applied before test
commit 67f7351c5c365c38218853f1faf9a8124894b442
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow full binary path to cassandra to be configured through env variables
    
    The current hard-coded values won't work for other distributions.
    
    This also fixes an issue on log path inexistence which raises.

commit 1a8924be8f7da912cc4bcd6ce2f749b9454dcbad
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sat Jul 11 08:42:17 2020 +0200

    158: Make schema and migration converge so the migration works
    
    In the end, the order of the revision entry matters whether we select * or not.
    So the select must match the order defined in the revision_entry type type.
    Otherwise, a mismatch type error occurs [1]
    
    [1] psql:sql/upgrades/158.sql:74: ERROR: return type mismatch in function
    declared to return revision_entry

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/446/ for more details.

ardumont retitled this revision from tests: Allow full binary path to cassandra to be configured through env variables to tests: Allow cassandra full binary path to be configured through env variable.Jul 12 2020, 4:10 PM
ardumont updated this revision to Diff 12391.Jul 12 2020, 4:17 PM

Rework commit message and improve documentation sentence

Build is green

Patch application report for D3505 (id=12391)

Could not rebase; Attempt merge onto 9219a23606...

Updating 9219a236..1881560e
Fast-forward
 README.md                           |  4 +++-
 sql/upgrades/158.sql                | 10 +++++-----
 swh/storage/sql/40-swh-func.sql     | 10 +++++-----
 swh/storage/tests/test_cassandra.py | 10 +++++++---
 tox.ini                             |  1 +
 5 files changed, 21 insertions(+), 14 deletions(-)
Changes applied before test
commit 1881560e001cbd772fe26a98a14b99c0134e006b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow cassandra full binary path to be configured through env variable
    
    The current hard-coded value won't work for other distributions.
    
    This also fixes an issue on log path inexistence which raises.

commit 1a8924be8f7da912cc4bcd6ce2f749b9454dcbad
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sat Jul 11 08:42:17 2020 +0200

    158: Make schema and migration converge so the migration works
    
    In the end, the order of the revision entry matters whether we select * or not.
    So the select must match the order defined in the revision_entry type type.
    Otherwise, a mismatch type error occurs [1]
    
    [1] psql:sql/upgrades/158.sql:74: ERROR: return type mismatch in function
    declared to return revision_entry

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/447/ for more details.

ardumont edited the summary of this revision. (Show Details)Wed, Jul 15, 9:29 AM
olasd requested changes to this revision.Wed, Jul 15, 10:51 AM
olasd added a subscriber: olasd.

Thanks for this change! Can we make these variables CASSANDRA_BIN and CASSANDRA_LOG (or SWH_* to match what we did with kafka)? the backwards naming feels a bit weird

This revision now requires changes to proceed.Wed, Jul 15, 10:51 AM

Thanks for this change!

picture me happy you like it ;)

Can we make these variables CASSANDRA_BIN and CASSANDRA_LOG (or SWH_* to match what we did with kafka)? the backwards naming feels a bit weird

sure, i'll have a look at what we did there (because right now, it's escaping me ;)

ardumont updated this revision to Diff 12399.Wed, Jul 15, 11:53 AM
ardumont edited the summary of this revision. (Show Details)
  • Adapt according to review (rename env variables to SWH_CASSANDRA_* prefix)
  • Rework commit message to align with the better description message
ardumont edited the summary of this revision. (Show Details)Wed, Jul 15, 11:54 AM

Build is green

Patch application report for D3505 (id=12399)

Rebasing onto 1a8924be8f...

Current branch diff-target is up to date.
Changes applied before test
commit 6fc187c38f8a6a42e6a9eb4c79e8b7386debef85
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow cassandra full binary path to be configured through env variable
    
    The current hard-coded value won't work for other distributions not relying on
    standard conventions (e.g. nixos...).
    
    This keeps the original behavior and only allow to diverge based on the
    environment variable SWH_CASSANDRA_BIN.
    
    This also:
    - fixes an issue on log path inexistence which raises.
    - renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for
      consistency)

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/448/ for more details.

ardumont updated this revision to Diff 12400.Wed, Jul 15, 12:09 PM
ardumont edited the summary of this revision. (Show Details)

Rework commit message and README sentence

ardumont retitled this revision from tests: Allow cassandra full binary path to be configured through env variable to tests: Allow cassandra binary path to be configured through env variable.Wed, Jul 15, 12:09 PM
ardumont edited the summary of this revision. (Show Details)

Build is green

Patch application report for D3505 (id=12400)

Rebasing onto 1a8924be8f...

Current branch diff-target is up to date.
Changes applied before test
commit f5811da184683402c8dec0721ce6db40b33325ab
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow cassandra binary path to be configured through env variable
    
    The current hard-coded value won't work for other distributions not relying on
    standard conventions (e.g. nixos...).
    
    This keeps the original behavior and only allow to diverge based on the
    environment variable SWH_CASSANDRA_BIN.
    
    This also:
    - fixes an issue on log path inexistence which raises.
    - renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for
      consistency)

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/449/ for more details.

olasd accepted this revision.Wed, Jul 15, 12:50 PM
This revision is now accepted and ready to land.Wed, Jul 15, 12:50 PM