Page MenuHomeSoftware Heritage

tests: Allow cassandra binary path to be configured through env variable
ClosedPublic

Authored by ardumont on Jul 12 2020, 4:01 PM.

Details

Summary

The current hard-coded value won't work for other distributions not relying on
standard conventions (e.g. nixos...).

This keeps the original behavior and only allow to diverge based on the
environment variable BIN_CASSANDRA [1]
This also:

  • fixes an issue on log path inexistence which raises.
  • renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for consistency)

Note:
Currently trying to run this with my nixos box and this minimum prevents it.

Test Plan

tox

Diff Detail

Repository
rDSTO Storage manager
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 13613
Build 20847: Phabricator diff pipeline on jenkinsJenkins console · Jenkins
Build 20846: arc lint + arc unit

Event Timeline

Build is green

Patch application report for D3505 (id=12389)

Could not rebase; Attempt merge onto 9219a23606...

Updating 9219a236..67f7351c
Fast-forward
 README.md                           |  4 +++-
 sql/upgrades/158.sql                | 10 +++++-----
 swh/storage/sql/40-swh-func.sql     | 10 +++++-----
 swh/storage/tests/test_cassandra.py | 10 +++++++---
 tox.ini                             |  1 +
 5 files changed, 21 insertions(+), 14 deletions(-)
Changes applied before test
commit 67f7351c5c365c38218853f1faf9a8124894b442
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow full binary path to cassandra to be configured through env variables
    
    The current hard-coded values won't work for other distributions.
    
    This also fixes an issue on log path inexistence which raises.

commit 1a8924be8f7da912cc4bcd6ce2f749b9454dcbad
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sat Jul 11 08:42:17 2020 +0200

    158: Make schema and migration converge so the migration works
    
    In the end, the order of the revision entry matters whether we select * or not.
    So the select must match the order defined in the revision_entry type type.
    Otherwise, a mismatch type error occurs [1]
    
    [1] psql:sql/upgrades/158.sql:74: ERROR: return type mismatch in function
    declared to return revision_entry

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/446/ for more details.

ardumont retitled this revision from tests: Allow full binary path to cassandra to be configured through env variables to tests: Allow cassandra full binary path to be configured through env variable.Jul 12 2020, 4:10 PM

Rework commit message and improve documentation sentence

Build is green

Patch application report for D3505 (id=12391)

Could not rebase; Attempt merge onto 9219a23606...

Updating 9219a236..1881560e
Fast-forward
 README.md                           |  4 +++-
 sql/upgrades/158.sql                | 10 +++++-----
 swh/storage/sql/40-swh-func.sql     | 10 +++++-----
 swh/storage/tests/test_cassandra.py | 10 +++++++---
 tox.ini                             |  1 +
 5 files changed, 21 insertions(+), 14 deletions(-)
Changes applied before test
commit 1881560e001cbd772fe26a98a14b99c0134e006b
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow cassandra full binary path to be configured through env variable
    
    The current hard-coded value won't work for other distributions.
    
    This also fixes an issue on log path inexistence which raises.

commit 1a8924be8f7da912cc4bcd6ce2f749b9454dcbad
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sat Jul 11 08:42:17 2020 +0200

    158: Make schema and migration converge so the migration works
    
    In the end, the order of the revision entry matters whether we select * or not.
    So the select must match the order defined in the revision_entry type type.
    Otherwise, a mismatch type error occurs [1]
    
    [1] psql:sql/upgrades/158.sql:74: ERROR: return type mismatch in function
    declared to return revision_entry

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/447/ for more details.

olasd requested changes to this revision.Jul 15 2020, 10:51 AM
olasd added a subscriber: olasd.

Thanks for this change! Can we make these variables CASSANDRA_BIN and CASSANDRA_LOG (or SWH_* to match what we did with kafka)? the backwards naming feels a bit weird

This revision now requires changes to proceed.Jul 15 2020, 10:51 AM

Thanks for this change!

picture me happy you like it ;)

Can we make these variables CASSANDRA_BIN and CASSANDRA_LOG (or SWH_* to match what we did with kafka)? the backwards naming feels a bit weird

sure, i'll have a look at what we did there (because right now, it's escaping me ;)

ardumont edited the summary of this revision. (Show Details)
  • Adapt according to review (rename env variables to SWH_CASSANDRA_* prefix)
  • Rework commit message to align with the better description message

Build is green

Patch application report for D3505 (id=12399)

Rebasing onto 1a8924be8f...

Current branch diff-target is up to date.
Changes applied before test
commit 6fc187c38f8a6a42e6a9eb4c79e8b7386debef85
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow cassandra full binary path to be configured through env variable
    
    The current hard-coded value won't work for other distributions not relying on
    standard conventions (e.g. nixos...).
    
    This keeps the original behavior and only allow to diverge based on the
    environment variable SWH_CASSANDRA_BIN.
    
    This also:
    - fixes an issue on log path inexistence which raises.
    - renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for
      consistency)

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/448/ for more details.

ardumont edited the summary of this revision. (Show Details)

Rework commit message and README sentence

ardumont retitled this revision from tests: Allow cassandra full binary path to be configured through env variable to tests: Allow cassandra binary path to be configured through env variable.Jul 15 2020, 12:09 PM
ardumont edited the summary of this revision. (Show Details)

Build is green

Patch application report for D3505 (id=12400)

Rebasing onto 1a8924be8f...

Current branch diff-target is up to date.
Changes applied before test
commit f5811da184683402c8dec0721ce6db40b33325ab
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Sun Jul 12 15:57:47 2020 +0200

    Allow cassandra binary path to be configured through env variable
    
    The current hard-coded value won't work for other distributions not relying on
    standard conventions (e.g. nixos...).
    
    This keeps the original behavior and only allow to diverge based on the
    environment variable SWH_CASSANDRA_BIN.
    
    This also:
    - fixes an issue on log path inexistence which raises.
    - renames the other env variable LOG_CASSANDRA to SWH_CASSANDRA_LOG (for
      consistency)

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/449/ for more details.

This revision is now accepted and ready to land.Jul 15 2020, 12:50 PM