diff --git a/.gitignore b/.gitignore index c023fa3..31b263c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,17 +1,17 @@ .bundle/ +coverage/ +doc/ +Gemfile.lock .idea/ -.project -.travis/secrets.tar -.vagrant/ -.yardoc/ *.iml +.librarian/ +log/ +pkg/ +.project +Puppetfile.lock +spec/fixtures/ *.swn *.swo *.swp -Gemfile.lock -coverage/ -doc/ -pkg/ -spec/fixtures/ vendor/ -log/ +.yardoc/ diff --git a/.rubocop.yml b/.rubocop.yml index c058c03..d6edbaa 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -1,5 +1,7 @@ --- +Metrics/BlockLength: + Enabled: false Metrics/LineLength: Enabled: false Style/FrozenStringLiteralComment: Enabled: false diff --git a/.travis.yml b/.travis.yml index 8540920..948d8df 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,27 +1,29 @@ --- env: - "PUPPET_VERSION='~> 4.5' BEAKER_set=centos6" - "PUPPET_VERSION='~> 4.5' BEAKER_set=centos7" - "PUPPET_VERSION='~> 4.5' BEAKER_set=debian7" - "PUPPET_VERSION='~> 4.5' BEAKER_set=debian8" - "PUPPET_VERSION='~> 4.5' BEAKER_set=ubuntu1204" - "PUPPET_VERSION='~> 4.5' BEAKER_set=ubuntu1404" - "PUPPET_VERSION='~> 4.5' BEAKER_set=ubuntu1604" language: ruby rvm: - 2.3.1 cache: bundler before_install: rm Gemfile.lock || true bundler_args: --without development dist: trusty sudo: required script: - "bundle exec puppet --version" - "bundle exec rake metadata_lint" - "bundle exec rubocop" - "bundle exec rake lint" - "bundle exec rake validate" - "bundle exec rake spec SPEC_OPTS='--format documentation'" - - "bundle exec rake acceptance" + - travis_wait 40 "bundle exec rake acceptance" notifications: email: false +matrix: + fast_finish: true diff --git a/CHANGELOG.md b/CHANGELOG.md index 158de2f..f43fd4a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,1080 +1,1092 @@ # Change Log for Puppet Module locp-cassandra +##2017-01-21 - Release 2.3.0 ([diff](https://github.com/locp/cassandra/compare/2.2.1...2.3.0)) + +New attribute to `::cassandra` class: + +* `baseline_settings` + +New defined types: + +* `cassandra::system::swapoff` +* `cassandra::system::sysctl` +* `cassandra::system::transparent_hugepage` + ##2016-12-08 - Release 2.2.1 ([diff](https://github.com/locp/cassandra/compare/2.2.0...2.2.1)) ### Summary Non-functional improvements and some documentation fixes. ### Improvements * All beaker spec tests have been migrated from AWS/EC2 to Docker. * Installed Python 2.7 onto the CentOS 6 docker image allowing the full test suite to be run against that operating system. * Some documentation improvements and fixed some typos. * The spec tests will now fail if there is not 100% coverage. ##2016-12-01 - Release 2.2.0 ([diff](https://github.com/locp/cassandra/compare/2.1.2...2.2.0)) ### Summary At long last we have a defined type for handling permissions (something that we have been promising since May). Added some more facts for assisting in JVM tuning. Also fixed an error that was happening when attempting to build the project. ### Features * Added the `permissions` attribute to the `cassandra::schema` class. * Added the `cassandra::schema::permission` ### Bugfixes * Some refactoring of the ruby code since the release of Rubocop 0.46.0 which was failing code that previously had passed OK. ##2016-11-02 - Release 2.1.2 ([diff](https://github.com/locp/cassandra/compare/2.1.1...2.1.2)) ### Summary Basically a non-functional release to reduce some of the technical debt from the 2.0.0 release. Also some minor improvements to the documentation. ##2016-11-19 - Release 1.26.1 ([diff](https://github.com/locp/cassandra/compare/1.26.0...1.26.1)) ### Summary Retro-fitted a bug that was fixed in 2.1.1 into the 1.X.Y version of the module. ### Bugfixes * A hotfix for a bug in `cassandra::schema` that has been around since version 1.18.0 of this module. ##2016-11-02 - Release 2.1.1 ([diff](https://github.com/locp/cassandra/compare/2.1.0...2.1.1)) ### Summary A hotfix for a bug in `cassandra::schema` that has been around since version 1.18.0 of this module. ### Bugfixes * Fixed a bug where cql_types would not be created unless keyspaces were also provided. * Fixed a minor typo in the README. ##2016-10-29 - Release 2.1.0 ([diff](https://github.com/locp/cassandra/compare/2.0.2...2.1.0)) ### Summary A mixture of new features, a new style of documentation and other improvements. Also ran the [Sonar-Puppet](https://github.com/iwarapter/sonar-puppet) plugin against the code to improve the code quality. ### Features * The following facts are now available: * cassandramajorversion * cassandraminorversion * cassandrapatchversion * cassandrarelease * The following attributes are now available: * cassandra::commitlog_directory * cassandra::commitlog_directory_mode * cassandra::data_file_directories * cassandra::data_file_directories_mode * cassandra::hints_directory * cassandra::hints_directory_mode * cassandra::saved_caches_directory * cassandra::saved_caches_directory_mode * cassandra::systemctl ### Bugfixes * Resolved all issues identified by Sonar-Puppet from 38 (2 major, 36 minor). In doing so, this reduced the technical debt ration from 0.9% to 0.0%. ### Improvements * The reference documentation is now processed via http://locp.github.io/cassandra * Made a start on refactoring the spec/unit tests so that there is less duplication of code. ##2016-10-12 - Release 2.0.2 ([diff](https://github.com/locp/cassandra/compare/2.0.1...2.0.2)) ### Summary This is a non-functional patch release that fixes some issues in documentation, and fixes some bugs and makes some improvements in our testing framework. ### Improvements * Ensure there is a framework for supporting the legacy 1.X.Y. module until the end of 2016. * A more obvious test rig for the beaker/acceptance tests on TravisCI. ### Bugfixes * Corrected misleading documentation. * Ensure legacy test builds still work against Ruby 1.9.3. * Corrected deprecated tags in the module metadata. ##2016-10-08 - Release 1.26.0 ([diff](https://github.com/locp/cassandra/compare/1.25.2...1.26.0)) ### Improvements * The `cassandra::seeds` paramater can now be provided as an array or as a string. # Bugfixes * Ensure legacy builds still work on TravisCI. ##2016-10-01 - Release 2.0.1 ([diff](https://github.com/locp/cassandra/compare/2.0.0...2.0.1)) ### Summary A non-functional release containing some improvements and documentation corrections. ### Improvements * Corrected some out of date documentation. * Removed some deprecated files. * Regained 100% spec test coverage. ##2016-09-29 - Release 2.0.0 ([diff](https://github.com/locp/cassandra/compare/1.26.0...2.0.0)) ### Summary This is a major release and will more than likely break your existing manifest until you alter your code. Please see the [Upgrading](https://github.com/locp/cassandra/tree/release/274/v2.0.0#upgrading) section of the README for further details. * The method of passing parameters to Cassandra has been made more flexible. * Support for Ubuntu 16.04. * Changes to how `service_ensure` is handled. * `cassandra::file` is now a define, not a class (as it should have been). ##2016-09-03 - Release 1.25.2 ([diff](https://github.com/locp/cassandra/compare/1.25.1...1.25.2)) ### Summary The `service_systemd` attribute is now deprecated for the `cassandra`, `cassandra::datastax_agent` and `cassandra::opscenter` classes. See https://github.com/locp/cassandra/wiki/DEP-020 for more details. ##2016-08-14 - Release 1.25.1 ([diff](https://github.com/locp/cassandra/compare/1.25.0...1.25.1)) ### Summary Bugfix to ensure that the name of the PID file for the DataStax Agent service matches with the service name. ##2016-07-17 - Release 1.25.0 ([diff](https://github.com/locp/cassandra/compare/1.24.3...1.25.0)) ### Summary A feature release. ### Features * The `cassandra` class now has a `disk_access_mode` attribute. ##2016-06-26 - Release 1.24.3 ([diff](https://github.com/locp/cassandra/compare/1.24.2...1.24.3)) ### Summary A couple of non-functional bug fixes. ### Bugfixes * The new version of RuboCop flagged code changes required in the Vagrantfile. * The new version of Puppet Lint flagged code changes required in cassandra::java. ##2016-06-20 - Release 1.24.2 ([diff](https://github.com/locp/cassandra/compare/1.24.1...1.24.2)) ### Summary A non-functional bug fix. ### Bugfixes * The release of the specinfra gem (2.59.1) caused problems with our build. Pinned now to 2.59.0. ##2016-06-18 - Release 1.24.1 ([diff](https://github.com/locp/cassandra/compare/1.24.0...1.24.1)) ### Summary Really only affects CentOS 7 as all the other supported operating systems do not require the systemd file to be installed. ### Bugfixes * Change the way that systemd shuts Cassandra down. ##2015-05-25 - Release 1.24.0 ([diff](https://github.com/locp/cassandra/compare/1.23.1...1.24.0)) ### Summary A Debian-centric release. Debian 8 is now supported and a workaround for [CASSANDRA-2356](https://issues.apache.org/jira/browse/CASSANDRA-2356) (which only affects Debian. ### Features * New attributes to the `::cassandra` class are: * cassandra_2356_sleep_seconds * config_path_parents * rackdc_tmpl ### Bugfixes * CASSANDRA-2356 ### Improvements * Debian 8 is now part of the acceptance testing release process. ##2016-05-10 - Release 1.23.1 ([diff](https://github.com/locp/cassandra/compare/1.23.0...1.23.1)) ### Summary A minor bugfix that fixes a problem that would only affect module developers, not users of the module. ### Bugfixes * Rubocop 0.4.0 is stricter so implemented changes that were flagged by it. ##2016-05-07 - Release 1.23.0 ([diff](https://github.com/locp/cassandra/compare/1.22.1...1.23.0)) ### Summary A couple of new features, a non-functional improvement and a bugfix. ### Features * Added the cassandra::file class. * Added the following attributes to the cassandra::schema class: * cqlsh_client_tmpl * cqlsh_client_config ### Bugfixes * Fixed a daft error that stopped beaker tests running on CentOS 7. ### Improvements * Improved how fast the beaker/acceptance tests are run from TravisCI. ##2016-05-03 - Release 1.22.1 ([diff](https://github.com/locp/cassandra/compare/1.22.0...1.22.1)) ### Summary A non-functional release for a minor improvement and a bug fix. ### Bugfixes * Fixes a problem with resource ordering if the service_refresh attribute is set to false. ### Improvements * Test coverage in Coveralls is now back at 100% (https://coveralls.io/github/locp/cassandra). ##2016-04-25 - Release 1.22.0 ([diff](https://github.com/locp/cassandra/compare/1.21.0...1.22.0)) ### Summary Two new features. ### Features * Allow the creation and dropping of users with the following: * A new defined type cassandra::schema::user * The new attribute cassandra::schema::users * Allow the manipulation of the environment file with the new class cassandra::env. ##2016-04-20 - Release 1.21.0 ([diff](https://github.com/locp/cassandra/compare/1.20.0...1.21.0)) ### Summary Some minor bug fixes. Also some features and improvements that allow acceptance tests to be run against Cassandra 3. ### Features * Added the following attributes to cassandra::java: * aptkey * aptsource * yumrepo ### Bugfixes * Corrected some errors in the README and CHANGELOG. * Fixed problems with TravisCI builds. * Fixed problems with CircleCI builds. ### Improvements * Acceptance testing of Cassandra 3. ##2016-04-13 - Release 1.20.0 ([diff](https://github.com/locp/cassandra/compare/1.19.0...1.20.0)) ### Summary A minor release for client requirements. ### Features * Added the compaction_large_partition_warning_threshold_mb and memtable_allocation_type attributes to the cassandra class. ##2016-04-06 - Release 1.19.0 ([diff](https://github.com/locp/cassandra/compare/1.18.1...1.19.0)) ### Summary A new template attribute and a couple of bug fixes. ### Features * The hints_directory attribute has been added to the cassandra class for placing into the template. ### Bugfixes * Some documentation errors that had been identified have been resolved. * A problem with OpsCenter and systemd has been resolved. So far only CentOS 7 from the supported operating systems has been identified as being required to use systemd. ##2016-03-27 - Release 1.18.1 ([diff](https://github.com/locp/cassandra/compare/1.18.0...1.18.1)) ### Summary Bug fixes and some minor and non-functional improvements. ### Bugfixes * Resource ordering clarified in the cassandra::schema class. * The cqlsh command now attempts to connect to rpc_address, not the listen_address. ### Improvements * Optimised the CircleCI build process. * Made some changes to the documentation to better reflect the new functionality delivered in 1.18.0. ##2016-03-26 - Release 1.18.0 ([diff](https://github.com/locp/cassandra/compare/1.17.0...1.18.0)) ### Summary Some more functionality and sub-classes for cassandra::schema. Also some code and pipe-line improvements and a couple of bug fixes. ### Features * The cassandra::schema class now has the following additional attributes: * cql_types * indexes * tables There are also corresponting defined types for those attributes: * cassandra::schema::cql_type * cassandra::schema::index * cassandra::schema::table * The cassandra::opscenter class now has a new attribute called ldap_group_search_filter_with_dn. ### Bugfixes * In the firewalls class, an OpsCenter server also needs to connect to the Cassandra node it is monitoring as a client. The ports have been adjusted to allow this to happen. * rspec-puppet version 2.4.0 was breaking our builds so pegged ourselves to 2.3.2. ### Improvements * Some minor and non-functional improvements to the build pipe-line. * Carried out an audit using the Puppet plugin for SonarQube. The number of issues has been reduced from 227 (13 major, 214 minor) to 4 major issues of which 2 are false positives. The remaining issues will be resolved in 2.0.0. ##2016-03-22 - Release 1.17.0 ([diff](https://github.com/locp/cassandra/compare/1.16.0...1.17.0)) ### Summary Another small change that is an emergency requirement for a client. ### Features * Added the `storage_keyspace` attribute to the cassandra::datastax_agent class. ##2016-03-15 - Release 1.16.0 ([diff](https://github.com/locp/cassandra/compare/1.15.1...1.16.0)) ### Summary A smaller release than usual, but containing changes required for a client. ### Features * Added the `hosts` attribute to the cassandra::datastax_agent class. ### Bugfixes * The **address.yaml** file for the DataStax agent is now owned by the cassandra user. ##2016-03-11 - Release 1.15.1 ([diff](https://github.com/locp/cassandra/compare/1.15.0...1.15.1)) ### Summary A small improvement. ### Improvements * Clarified CQL and Python versions. ##2016-03-10 - Release 1.15.0 ([diff](https://github.com/locp/cassandra/compare/1.14.2...1.15.0)) ### Summary A rather large release. Minor (non-functional) fixes to the production pipeline and new features. ### Features * A new class `cassandra::schema` allows the creation and dropping of keyspaces. * Added the `additional_lines` attribute to the `cassandra` class. * Added the `service_systemd` attribute to the `cassandra::opscenter` class. * Allow the systemd template sources to be specified by the user. This is with the `service_systemd_tmpl` attribute to the following classes: * cassandra * cassandra::datastax_agent * cassandra::opscenter * Added another template file for `cassandra` => `service_systemd_tmpl` that is suitable for Cassandra 2.0. ### Bugfixes * Worked around a problem with rake-11 in the Gemfile. * Fixed a problem in the CircleCI configuration. ### Improvements * Renamed the `ensure` attribute to `package_ensure` so that it is more in line with how it's called in other modules. This was done on the the following classes: * cassandra::optutils * cassandra::opscenter * cassandra::opscenter::pycrypto * cassandra::java * Updated the AWS AMI for the TravisCI/AWS acceptance testing to hopefully speed up builds a bit. ##2016-02-29 - Release 1.14.2 ([diff](https://github.com/locp/cassandra/compare/1.14.1...1.14.2)) ### Summary A small bugfix release. ### Bugfixes * Removed `Restart=always` from the Systemd configuration files. ##2016-02-27 - Release 1.14.1 ([diff](https://github.com/locp/cassandra/compare/1.14.0...1.14.1)) ### Summary A non-functional release of improvements and a bugfix. ### Bugfixes * If a systemd service file is created or updated, then systemctl daemon-reload is now be executed. ### Improvements * The workflow for building the module has been improved to include: * Automatic integration of improvements and bugfixes into release candidates. * Testing of release candidates includes acceptance (beaker) as well as unit (spec) tests. * The refactoring of the majority of the Ruby code used to test and build this module so that it is hopefully more readable and easier to maintain. * The automation of what had previously been manual steps when building a release. ##2016-02-19 - Release 1.14.0 ([diff](https://github.com/locp/cassandra/compare/1.13.0...1.14.0)) ### Summary A minor release with one of each of a feature, bug fix and improvement. ### Features * Added the orbited_longpoll attribute to the cassandra::opscenter class. ### Bugfixes * Fixed a problem with the DataStax agent and systemd. ### Improvements * Refactored the contributors section of the README. ##2016-02-14 - Release 1.13.0 ([diff](https://github.com/locp/cassandra/compare/1.12.2...1.13.0)) ### Summary A mixed back of new features in the shape of attributes for the cassandra, cassandra::datastax_agent and cassandra::opscenter classes. A couple of bug fixes and some non-functional improvements. ### Features * Added the thrift_framed_transport_size_in_mb attribute to ::cassandra. * Added the following attributes to ::cassandra::datastax_agent: * async_pool_size * async_queue_size * service_systemd * Added the config_purge attribute to ::cassandra::opscenter. ### Bugfixes * Removed incorrect puppet code from the README examples. * Fixed a problem with the beaker 2.34.0 gem which was causing problems during automated acceptance testing. ### Improvements * Changed the AWS instance type used by the TravisCI triggered acceptance tests from a c3.xlarge to c4.xlarge. * Merged the acceptance tests virtual nodes into family specific node sets. * Refactored the Gemfile. * Changed references in the documentation to parameters to refer to attributes as that is more Puppet-like. * Changed the format of the contributers section. ##2016-02-12 - Release 1.12.2 ([diff](https://github.com/locp/cassandra/compare/1.12.1...1.12.2)) ### Summary More bug fixes. ### Bugfixes * Fixed a problem with the Red Hat family and systemd not starting the service and reporting all service stops as failures, regardless of if they were or not. ##2016-02-08 - Release 1.12.1 ([diff](https://github.com/locp/cassandra/compare/1.12.0...1.12.1)) ### Summary This is a non-functional release. Some bug fixes and release improvements. ### Bugfixes * Completed documentation for attributes. This was missing for the `inter_dc_stream_throughput_outbound_megabits_per_sec` and `stream_throughput_outbound_megabits_per_sec` options. * Corrected the ownership and directories for the OpsCenter configuration. ### Improvements * Nightly build created so that issues similar to those found in issues #136 and #157 can be caught quicker. ##2016-01-27 - Release 1.12.0 ([diff](https://github.com/locp/cassandra/compare/1.11.0...1.12.0)) ### Summary A new feature in the cassandra::datastax_agent class, a minor bug fix and integration with CircleCI. ### Features * There is now an agent_alias attribute for the cassandra::datastax_agent class. ### Bugfixes * Unit tests were failing due to problems with the puppet-3.8.5 gem. ### Improvements * In addition to TravisCI, the build process is now integrated with [CircleCI](https://circleci.com/gh/locp/cassandra). ##2016-01-01 - Release 1.11.0 ([diff](https://github.com/locp/cassandra/compare/1.10.0...1.11.0)) ### Summary New features added to the main class. Also some non-functional improvements. ### Features * The addition of the listen_interface and rpc_interface attributes to the main class. ### Improvements * Added more detail to the attributes to the main class in the README. * Improved the module metadata. * Clarified private defined types with the private subclass. * The test coverage in release 1.10.0 dropped to 99.09%. Got it back to 100% in this release. ##2015-12-19 - Release 1.10.0 ([diff](https://github.com/locp/cassandra/compare/1.9.2...1.10.0)) ### Summary A feature release with minor improvements. ### Features * Added the ability to allow setting the local_interface for the DataStax agent configuration. * Allow the service provider to be specified for the Cassandra, DataStax agent and OpsCenter services with the service_provider attribute. * Optionally allow a systemd system file be inserted with the cassandra::service_systemd attribute. ### Improvements * Allow the files resources specified with in the cassandra attributes: * commitlog_directory * data_file_directories * saved_caches_directory To co-exist with file resources with the same name. ##2015-11-21 - Release 1.9.2 ([diff](https://github.com/locp/cassandra/compare/1.9.1...1.9.2)) ### Summary A bug fix release that deals with some problems with Cassandra 3. ### Bugfixes * Attempt to mitigate against problems with Debian attempting to install Cassandra 3 when installing the dsc22 package. * Also reverted the project home to the GitHub project page. ##2015-11-09 - Release 1.9.1 ([diff](https://github.com/locp/cassandra/compare/1.9.0...1.9.1)) ### Summary A bug fix release. ### Bugfixes * The default value for the permissions mode of the Cassandra configuration file were far too open. Changed from 0666 to 0644. ##2015-10-25 - Release 1.9.0 ([diff](https://github.com/locp/cassandra/compare/1.8.1...1.9.0)) ### Summary Added more features for the configuration of Cassandra, some improvements to the testing carried out before a release and a minor correction to the change log documentation. ### Features * The following attributes have been added to the ::cassandra class to be configured into the configuration file: * client_encryption_algorithm * client_encryption_cipher_suites * client_encryption_protocol * client_encryption_require_client_auth * client_encryption_store_type * client_encryption_truststore * client_encryption_truststore_password * counter_cache_size_in_mb * index_summary_capacity_in_mb * key_cache_save_period * key_cache_keys_to_save * seed_provider_class_name * server_encryption_algorithm * server_encryption_cipher_suites * server_encryption_protocol * server_encryption_require_client_auth * server_encryption_store_type Please see the README file for more details. ### Bugfixes * Corrected an incorrect date (typo) in this document. ### Improvements * There is now an automated test to mitigate the risk of unnecessarily refreshes of the Cassandra service due to non-functional changes to the configuration file. ##2015-10-14 - Release 1.8.1 ([diff](https://github.com/locp/cassandra/compare/1.8.0...1.8.1)) ### Summary A minor bug fix. ### Bugfixes * Fixed an edge case issue concerning users that may have been using the fail_on_non_supported_os before it was fixed in 1.8.0. ##2015-10-06 - Release 1.8.0 ([diff](https://github.com/locp/cassandra/compare/1.7.1...1.8.0)) ### Summary Some new features a minor bug fix and some non-functional improvements. ### Features * Added the service_refresh and config_file_mode attributes to the Cassandra class. ### Bugfixes * The name of the fail_on_non_supported_os attribute has been corrected. ### Improvements * Automated acceptance tests in preparation for a release now run faster. ##1015-10-01 - Release 1.7.1 ([diff](https://github.com/locp/cassandra/compare/1.7.0...1.7.1)) ### Summary A minor bug fix that incorrctly gave a failed build status for the module. ### Bugfixes * Fixed a problem that was showing the status of the module build as an error since the release of the fog-google gem version 0.1.1. ##2015-10-01 - Release 1.7.0 ([diff](https://github.com/locp/cassandra/compare/1.6.0...1.7.0)) ### Summary * Corrected a bug in how commitlog_sync has handled by Cassandra. * Some non-functional improvements * Additional features for the cassandra::datastax_repo class. ### Features * Added the commitlog_segment_size_in_mb attribute to the cassandra class. * Added the following fields to the cassandra::datastax_repo class: * descr * key_id * key_url * pkg_url * release This should make the configuring of repositories more flexible. ### Bugfixes * Fixed a bug in how the commitlog_sync and the attributes that are associated with it are handled ### Improvements The following non-functional improvements were implemented: * Added tags to the module metadata. * Migrated the acceptance tests from Vagrant to Docker. The associated improvements to performance means that more rigorous acceptance tests can be applied in a shorter time. For the first time as well, they are visible on Travis. ##2015-09-23 - Release 1.6.0 ([diff](https://github.com/locp/cassandra/compare/1.5.0...1.6.0)) ### Summary More attributes for ::cassandra and ::cassandra::datastax_agent. Also some non-functional improvements in the automated unit tests. ### Features * The JAVA_HOME can now be set for the datastax_agent (see the cassandra::datastax_agent => java_home attribute). * The file mode for the directories can now be specified for the commitlog_directory, data_file_directories and the saved_caches_directory in the cassandra class. ### Improvements * Uncovered resources in the unit testing are now tested. ##2015-09-21 - Release 1.5.0 ([diff](https://github.com/locp/cassandra/compare/1.4.2...1.5.0)) ### Summary More attributes have been added that can be configured into the cassandra.yaml file. ### Features * The following attributes to the cassandra class can be configured into the cassandra configuration: * broadcast_address * broadcast_rpc_address * commitlog_sync * commitlog_sync_batch_window_in_ms * commitlog_total_space_in_mb * concurrent_compactors * counter_cache_keys_to_save * file_cache_size_in_mb * initial_token * inter_dc_stream_throughput_outbound_megabits_per_sec * internode_authenticator * internode_recv_buff_size_in_bytes * internode_send_buff_size_in_bytes * memory_allocator * memtable_cleanup_threshold * memtable_flush_writers * memtable_heap_space_in_mb * memtable_offheap_space_in_mb * native_transport_max_concurrent_connections * native_transport_max_concurrent_connections_per_ip * native_transport_max_frame_size_in_mb * native_transport_max_threads * permissions_update_interval_in_ms * phi_convict_threshold * request_scheduler_options_default_weight * request_scheduler_options_throttle_limit * row_cache_keys_to_save * rpc_max_threads * rpc_min_threads * rpc_recv_buff_size_in_bytes * rpc_send_buff_size_in_bytes * streaming_socket_timeout_in_ms * stream_throughput_outbound_megabits_per_sec ### Improvements * Clarity of changes per release in the change log (this document). ##2015-09-15 - Release 1.4.2 ([diff](https://github.com/locp/cassandra/compare/1.4.1...1.4.2)) ### Summary Fixed a problem identified whilst releasing 1.4.1 and a bug fixed by a contributed pull request. ### Features * n/a ### Bugfixes * Fixed a problem with the acceptance tests. * The datastax-agent service is restarted if the package is updated. ### Improvements * n/a ##2015-09-15 - Release 1.4.1 ([diff](https://github.com/locp/cassandra/compare/1.4.0...1.4.1)) ### Summary This release fixes a minor bug (possibly better described as a typing mistake) and makes some non-functional improvements. It also allows the user to override the default behaviour of failing on a non-supported operating system. ### Features * A new flag called `fail_on_non_suppoted_os` has been added to the `cassandra` class and can be set to **false** so that an attempt can be made to use this module on an operating system that is not in the Debian or Red Hat families. ### Bugfixes * Changed the default value for the `package_name` of the `cassandra::optutils` class from `'undef'` to *undef*. ### Improvements * Clarified the expectations of submitted contributions. * Unit test improvements. ##2015-09-10 - Release 1.4.0 ([diff](https://github.com/locp/cassandra/compare/1.3.7...1.4.0)) * Ensured that directories specified in the directory attributes are controlled with file resources. * Added the following attributes to the cassandra.yml file: * batchlog_replay_throttle_in_kb * cas_contention_timeout_in_ms * column_index_size_in_kb * commit_failure_policy * compaction_throughput_mb_per_sec * counter_cache_save_period * counter_write_request_timeout_in_ms * cross_node_timeout * dynamic_snitch_badness_threshold * dynamic_snitch_reset_interval_in_ms * dynamic_snitch_update_interval_in_ms * hinted_handoff_throttle_in_kb * index_summary_resize_interval_in_minutes * inter_dc_tcp_nodelay * max_hints_delivery_threads * max_hint_window_in_ms * permissions_validity_in_ms * range_request_timeout_in_ms * read_request_timeout_in_ms * request_scheduler * request_timeout_in_ms * row_cache_save_period * row_cache_size_in_mb * sstable_preemptive_open_interval_in_mb * tombstone_failure_threshold * tombstone_warn_threshold * trickle_fsync * trickle_fsync_interval_in_kb * truncate_request_timeout_in_ms * write_request_timeout_in_ms ##2015-09-08 - Release 1.3.7 ([diff](https://github.com/locp/cassandra/compare/1.3.6...1.3.7)) * Made the auto_bootstrap attribute available. ##2015-09-03 - Release 1.3.6 ([diff](https://github.com/locp/cassandra/compare/1.3.5...1.3.6)) * Fixed a bug, now allowing the user to set the enabled state of the Cassandra service. * More cleaning up of the README and more links in that file to allow faster navigation. ##2015-09-01 - Release 1.3.5 ([diff](https://github.com/locp/cassandra/compare/1.3.4...1.3.5)) * Fixed a bug, now allowing the user to set the running state of the Cassandra service. * More automated testing with spec tests. * A refactoring of the README. ##2015-08-28 - Release 1.3.4 ([diff](https://github.com/locp/cassandra/compare/1.3.3...1.3.4)) * Minor corrections to the README. * The addition of the storage_cassandra_seed_hosts attribute to cassandra::opscenter::cluster_name which is part of a bigger part of work but is urgently require by a client. ##2015-08-27 - Release 1.3.3 ([diff](https://github.com/locp/cassandra/compare/1.3.2...1.3.3)) * Corrected dependency version for puppetlabs-apt. ##2015-08-26 - Release 1.3.2 ([diff](https://github.com/locp/cassandra/compare/1.3.1...1.3.2)) * Fixed bug in cassandra::opscenter::cluster_name. * Fixed code in cassandra::firewall_ports::rule to avoid deprecation warnings concerning the use of puppetlabs-firewall => port. * Added more examples to the README ##2015-08-22 - Release 1.3.1 ([diff](https://github.com/locp/cassandra/compare/1.3.0...1.3.1)) This was mainly a non-functional change. The biggest thing to say is that Debian 7 is now supported. ##2015-08-19 - Release 1.3.0 ([diff](https://github.com/locp/cassandra/compare/1.2.0...1.3.0)) * Allow additional TCP ports to be specified for the host based firewall. * Fixed a problem where the client subnets were ignored by the firewall. * Added more automated testing. * Continued work on an ongoing improvement of the documentation. * Added the ability to set the DC and RACK in the snitch properties. ##2015-08-10 - Release 1.2.0 ([diff](https://github.com/locp/cassandra/compare/1.1.0...1.2.0)) * Added the installation of Java Native Access (JNA) to cassandra::java * For DataStax Enterprise, allow the remote storage of metric data with cassandra::opscenter::cluster_name. ##2015-08-03 - Release 1.1.0 ([diff](https://github.com/locp/cassandra/compare/1.0.1...1.1.0)) * Provided the cassandra::firewall_ports class. * All OpsCenter options are now configurable in opscenterd.conf. * ssl_storage_port is now configurable. ##2015-07-27 - Release 1.0.1 ([diff](https://github.com/locp/cassandra/compare/1.0.0...1.0.1)) * Provided a workaround for [CASSANDRA-9822](https://issues.apache.org/jira/browse/CASSANDRA-9822). ##2015-07-25 - Release 1.0.0 ([diff](https://github.com/locp/cassandra/compare/0.4.3...1.0.0)) * Changed the default installation from Cassandra 2.1 to 2.2. * Fixed a bug that arose when the cassandra config_path was set. * Created a workaround for [PUP-3829](https://tickets.puppetlabs.com/browse/PUP-3829). * Minor changes to the API (see the Upgrading section of the README). * Allow a basic installation of OpsCenter. ##2015-07-18 - Release 0.4.3 ([diff](https://github.com/locp/cassandra/compare/0.4.2...0.4.3)) * Module dependency metadata was too strict. ##2015-07-16 - Release 0.4.2 ([diff](https://github.com/locp/cassandra/compare/0.4.1...0.4.2)) * Some minor documentation changes. * Fixed a problem with the module metadata that caused Puppetfile issues. * Integrated with Coveralls (https://coveralls.io/github/locp/cassandra). * Removed the deprecated config and install classes. These were private so there is no change to the API. ##2015-07-14 - Release 0.4.1 ([diff](https://github.com/locp/cassandra/compare/0.4.0...0.4.1)) * Fixed a resource ordering problem in the cassandra::datastax class. * Tidied up the documentation a bit. * Some refactoring of the spec tests. ##2015-07-12 - Release 0.4.0 ([diff](https://github.com/locp/cassandra/compare/0.3.0...0.4.0)) ### Summary * Some major changes to the API on how Java, the optional Cassandra tools and the DataStax agent are installed. See the Upgrading section of the README file. * Allowed the setting of the *stomp_interface* for the DataStax agent. * Non-functionally, we have integrated with Travis CI (see https://travis-ci.org/locp/cassandra for details) and thanks to those guys for providing such a neat service. * More spec tests. ##2015-06-27 - Release 0.3.0 ([diff](https://github.com/locp/cassandra/compare/0.2.2...0.3.0)) ### Summary * Slight changes to the API. See the Upgrading section of the README file for full details. * Allow for the installation of the DataStax Agent. * Improved automated testing (and fixed some bugs along the way). * Confirmed Ubuntu 12.04 works OK with this module. * A Cassandra 1.X template has been provided. * Some smarter handling of the differences between Ubuntu/Debian and RedHat derivatives. ##2015-06-17 - Release 0.2.2 ([diff](https://github.com/locp/cassandra/compare/0.2.1...0.2.2)) ### Summary A non-functional change to change the following: * Split the single manifest into multiple files. * Implement automated testing. * Test on additional operating systems. ##2015-05-28 - Release 0.2.1 ([diff](https://github.com/locp/cassandra/compare/0.2.0...0.2.1)) ### Summary A non-functional change to fix puppet-lint problems identified by Puppet Forge. ##2015-05-28 - Release 0.2.0 ([diff](https://github.com/locp/cassandra/compare/0.1.0...0.2.0)) ### Summary Added more attributes and improved the module metadata. ##2015-05-26 - Release 0.1.0 ### Summary An initial release with **VERY** limited options. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2fa0fa1..4c62d55 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,168 +1,169 @@ # Contributing to the Module or Raising Issues ## Table of Contents 1. [Raising an Issue](#raising-an-issue) 1. [Contribtions](#contributions) * [Unit Tests](#unit-tests) * [Acceptance Tests](#acceptance-tests) 1. [Contributors](#contributors) 1. [Further Reading](#further-reading) ## Raising an Issue When raising an issue, please provide the following information: * The version of the locp-cassandra module that you are using. * The version of Cassandra that you are installing. * The operating system and release (output from `facter os` would be appropriate). * A sample of your manifest/profile that is calling the `cassandra` module. Feel free to obfuscate sections of the code that contain details that are confidential (e.g. passwords and other secrets). ## Contributions Contributions will be gratefully accepted. Please go to the project page, fork the project, make your changes locally and then raise a pull request. Details on how to do this are available at https://guides.github.com/activities/contributing-to-open-source. However, we do ask that at the very least, all items marked as **MUST** or **WON'T** in the list below are applicable: * Any new features (e.g. new resources or new attributes to existing resoures) **MUST** be fully documented . * Unit tests **MUST** be completing successfully. See [Unit Tests](#unit-tests) for more details. If your initial unit tests fail after a pull request and you need to fix them, simply change the code on your branch and push them to *origin* again as this will re-run the tests. It is not required to submit a new pull request. * Any new functionality or enhancements **SHOULD** be covered by unit/spec tests. If you are not comfortable with this, submit the PR anyway and we will fill in these gaps. You will most probably be asked to rebase your PR branch and then push again to register these changes. * If applicable, changes **COULD** be covered in beaker/acceptance tests. * Change **WON'T** break any functionality on any of the supported operating systems. ### Unit Tests First, you'll need to install the testing dependencies using [bundler](http://bundler.io). ```shell bundle install ``` To run all of the unit tests execute the following: ```shell bundle exec rake test ``` This should output something like the following: ``` Running RuboCop... Inspecting 24 files ........................ 24 files inspected, no offenses detected ---> syntax:manifests ---> syntax:templates ---> syntax:hiera:yaml /home/ben/.rvm/rubies/ruby-2.1.6/bin/ruby -I/home/ben/.rvm/gems/ruby-2.1.6/gems/rspec-core-3.5.4/lib:/home/ben/.rvm/gems/ruby-2.1.6/gems/rspec-support-3.5.0/lib /home/ben/.rvm/gems/ruby-2.1.6/gems/rspec-core-3.5.4/exe/rspec --pattern spec/\{aliases,classes,defines,unit,functions,hosts,integration,types\}/\*\*/\*_spec.rb --color [Coveralls] Set up the SimpleCov formatter. [Coveralls] Using SimpleCov's default settings. ....................................................... Finished in 7.86 seconds (files took 0.81841 seconds to load) 55 examples, 0 failures Total resources: 64 Touched resources: 64 Resource coverage: 100.00% [Coveralls] Outside the CI environment, not sending data. ``` Note that if you prefer, you can run the lint, syntax, and spec tests separately with individual commands: ```shell bundle exec rake metadata_lint bundle exec rake rubocop bundle exec rake lint bundle exec rake validate bundle exec rake spec ``` If in doubt, or you are stuck, please ask for help in the PR or via our [Gitter Room](https://gitter.im/locp/cassandra). The unit tests will also fail if the test coverage falls below 100%. ### Acceptance Tests These tests are more CPU intensive and are run via Docker. You will need to install further gems for this to work with the following command: ```shell bundle install --with acceptance ``` Then run the tests: ```shell bundle exec rake beaker:centos6 bundle exec rake beaker:centos7 bundle exec rake beaker:debian7 bundle exec rake beaker:debian8 bundle exec rake beaker:ubuntu1204 bundle exec rake beaker:ubuntu1404 bundle exec rake beaker:ubuntu1604 ``` ## Contributors **Release** | **PR/Issue** | **Contributer** ------------|-----------------------------------------------------|---------------------------------------------------- +2.3.0 | [Hiera documentation](https://github.com/locp/cassandra/pull/329) | [@tibers](https://github.com/tibers) 2.1.1 | [Correct cql_types conditional in cassandra::schema class](https://github.com/locp/cassandra/pull/325)| [@aaron-miller](https://github.com/aaron-miller) 2.0.2 | [locp/cassandra 2.0 example has two cassandra declarations](https://github.com/locp/cassandra/issues/291)| [@ericy-jana](https://github.com/ericy-jana) 2.0.0 | [Convert cassandra::file from a class to a define](https://github.com/locp/cassandra/issues/266)| [@stanleyz](https://github.com/stanleyz) 1.26.1 | [Correct cql_types conditional in cassandra::schema class](https://github.com/locp/cassandra/pull/325)| [@aaron-miller](https://github.com/aaron-miller) 1.25.2 | [Ubuntu 16 doesnt like the service file](https://github.com/locp/cassandra/issues/269)| [@ahharu](https://github.com/ahharu) 1.25.1 | [PID file name in systemd file for datastax-agent doesn't match PID file name in /etc/init.d/datastax-agent](https://github.com/locp/cassandra/issues/264)| [@pampelix](https://github.com/pampelix) 1.25.0 | [adding support for mmap](https://github.com/locp/cassandra/pull/261) | [@tibers](https://github.com/tibers) 1.24.0 | [remove varlib_dir require from data directory](https://github.com/locp/cassandra/pull/247) | [@ericy-jana](https://github.com/ericy-jana) 1.24.0 | [stage config files before installing cassandra package](https://github.com/locp/cassandra/pull/246) | [@ericy-jana](https://github.com/ericy-jana) 1.24.0 | [locp/cassandra starts two Cassandra processes](https://github.com/locp/cassandra/issues/245)| [@ericy-jana](https://github.com/ericy-jana) 1.23.0 | [attemping to realize client security](https://github.com/locp/cassandra/pull/235) | [@tibers](https://github.com/tibers) 1.22.1 | [#232 - add ordering to service definition](https://github.com/locp/cassandra/pull/233) | [@tibers](https://github.com/tibers) 1.22.1 | [module tries to start cassandra before it's installed](https://github.com/locp/cassandra/issues/232)| [@tibers](https://github.com/tibers) 1.21.0 | [Issue: Java version installed on Debian family by cassandra::java class #223](https://github.com/locp/cassandra/pull/226) | [@tibers](https://github.com/tibers) 1.20.0 | [Missing attributes in cassandra.yaml](https://github.com/locp/cassandra/issues/217)| [@samyray](https://github.com/samyray) 1.19.0 | [adding hints_directory for cassandra 3.x](https://github.com/locp/cassandra/pull/215) | [@tibers](https://github.com/tibers) 1.18.0 | [Added opscenter ldap option group_search_filter_with_dn](https://github.com/locp/cassandra/pull/203) | [@Mike-Petersen](https://github.com/Mike-Petersen) 1.15.0 | [creating vanilla cassandra 2.0.xx template](https://github.com/locp/cassandra/pull/189) | [@tibers](https://github.com/tibers) 1.14.0 | [Feature/opscenter orbited longpoll](https://github.com/locp/cassandra/pull/171) | [@jonen10](https://github.com/jonen10) 1.13.0 | [Allowing setting of async_pool_size and async_queue_size for the agent](https://github.com/locp/cassandra/pull/166) | [@Mike-Petersen](https://github.com/Mike-Petersen) 1.13.0 | [parameterized thrift_framed_transport_size_in_mb](https://github.com/locp/cassandra/pull/163) | [@VeriskPuppet](https://github.com/VeriskPuppet) 1.12.2 | [Error opening zip file or JAR manifest missing : /usr/sbin/../lib/jamm-0.3.0.jar](https://github.com/locp/cassandra/pull/165) | [@palmertime](https://github.com/palmertime) 1.12.0 | [Rename alias to agent_alias as alias is a reserved puppet word](https://github.com/locp/cassandra/pull/156) | [@stuartbfox](https://github.com/stuartbfox) 1.12.0 | [Added interfaces as an option](https://github.com/locp/cassandra/pull/153) | [@Mike-Petersen](https://github.com/Mike-Petersen) 1.10.0 | [Allowing setting of local_interface for the datastax agent configuration](https://github.com/locp/cassandra/pull/144) | [@Mike-Petersen](https://github.com/Mike-Petersen) 1.9.2 | [When installing cassandra dsc22 it tries to install cassandra 3.0.0 and as dependecy 2.2.3 is needed](https://github.com/locp/cassandra/issues/136)| [@mantunovic](https://github.com/mantunovic) 1.9.2 | [When installing cassandra dsc22 it tries to install cassandra 3.0.0 and as dependecy 2.2.3 is needed](https://github.com/locp/cassandra/issues/136)| [@al4](https://github.com/al4) 1.4.2 | [restart service if datastax agent package is upgraded](https://github.com/locp/cassandra/pull/110) | [@markasammut](https://github.com/markasammut) 1.4.0 | [allow batch_size_warn_threshold to be modified externally](https://github.com/locp/cassandra/pull/100) | [@markasammut](https://github.com/markasammut) 1.3.5 | [service_ensure unused](https://github.com/locp/cassandra/issues/93) | [@sampowers](https://github.com/sampowers) 1.3.3 | [Fails To Run With puppetlabs-apt v1.8.0](https://github.com/locp/cassandra/pull/87) | [@DylanGriffith](https://github.com/DylanGriffith) 0.4.2 | [Fix syntax in version_requirements](https://github.com/locp/cassandra/pull/34) | [@amosshapira](https://github.com/amosshapira) 0.3.0 | [Add a template for Cassandra 1.x compat](https://github.com/locp/cassandra/pull/11) | [@spredzy](https://github.com/Spredzy) ## Further Reading * *RSpec tests for your Puppet manifests* * *Beaker Info* diff --git a/Gemfile b/Gemfile index a6279af..beb1e2b 100644 --- a/Gemfile +++ b/Gemfile @@ -1,43 +1,43 @@ source ENV['GEM_SOURCE'] || 'https://rubygems.org' puppetversion = ENV.key?('PUPPET_VERSION') ? ENV['PUPPET_VERSION'] : ['>= 3.8'] group :test do gem 'coveralls', require: false gem 'facter', '>= 1.7.0' gem 'git', '1.3.0' gem 'hiera', require: false gem 'httparty', require: false gem 'metadata-json-lint', require: false gem 'puppet', puppetversion gem 'puppet-blacksmith', require: false gem 'puppet-lint', require: false gem 'puppet-strings', require: false gem 'puppetlabs_spec_helper', require: false gem 'rspec-puppet', '>= 2.3.2' gem 'rspec-puppet-utils', require: false gem 'rspec_junit_formatter', require: false gem 'rubocop-rspec', '1.4.1' if RUBY_VERSION < '2.2.0' gem 'travis', require: false gem 'travis-lint', require: false gem 'yard', require: false end group :acceptance do - gem 'beaker', require: false - gem 'beaker-puppet_install_helper', require: false - gem 'beaker-rspec', require: false - gem 'pry', require: false + gem 'beaker' + gem 'beaker-puppet_install_helper' + gem 'beaker-rspec' + gem 'pry' end group :development do gem 'notes', '~> 0.1.2' end # rspec must be v2 for ruby 1.8.7 if RUBY_VERSION >= '1.8.7' && RUBY_VERSION < '1.9' gem 'rake', '~> 10.0' gem 'rspec', '~> 2.0' else # rubocop requires ruby >= 1.9 gem 'rubocop' end diff --git a/LICENSE b/LICENSE index ab87265..5f939ec 100644 --- a/LICENSE +++ b/LICENSE @@ -1,201 +1,201 @@ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. - Copyright 2015, 2016 League of Crafty Programmers Ltd. + Copyright 2015 - 2017 League of Crafty Programmers Ltd. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. diff --git a/README.md b/README.md index 9626893..80d324b 100644 --- a/README.md +++ b/README.md @@ -1,600 +1,633 @@ # Cassandra [![Puppet Forge](http://img.shields.io/puppetforge/v/locp/cassandra.svg)](https://forge.puppetlabs.com/locp/cassandra) [![Github Tag](https://img.shields.io/github/tag/locp/cassandra.svg)](https://github.com/locp/cassandra) [![Build Status](https://travis-ci.org/locp/cassandra.png?branch=master)](https://travis-ci.org/locp/cassandra) [![Coverage Status](https://coveralls.io/repos/locp/cassandra/badge.svg?branch=master&service=github)](https://coveralls.io/github/locp/cassandra?branch=master) [![Join the chat at https://gitter.im/locp/cassandra](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/locp/cassandra?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![CircleCI](https://circleci.com/gh/locp/cassandra.svg?style=svg)](https://circleci.com/gh/locp/cassandra) [![Puppet Forge Downloads](http://img.shields.io/puppetforge/dt/locp/cassandra.svg)](https://forge.puppetlabs.com/locp/cassandra) [![Puppet Forge Endorsement](https://img.shields.io/puppetforge/e/locp/cassandra.svg)](https://forge.puppetlabs.com/locp/cassandra) ## Table of Contents 1. [Overview](#overview) 1. [Setup - The basics of getting started with Cassandra](#setup) * [What Cassandra affects](#what-cassandra-affects) * [Upgrading](#upgrading) * [Beginning with Cassandra](#beginning-with-cassandra) 1. [Usage - Configuration options and additional functionality](#usage) * [Setup a keyspace and users](#setup-a-keyspace-and-users) * [Create a Cluster in a Single Data Center](#create-a-cluster-in-a-single-data-center) * [Create a Cluster in Multiple Data Centers](#create-a-cluster-in-multiple-data-centers) * [DataStax Enterprise](#datastax-enterprise) 1. [Reference](#reference) 1. [Limitations - OS compatibility, etc.](#limitations) 1. [Development](#development) ## Overview A Puppet module to install and manage Cassandra, DataStax Agent & OpsCenter ## Setup ### What Cassandra affects #### What the Cassandra class affects * Installs the Cassandra package (default **cassandra22** on Red Hat and **cassandra** on Debian). * Configures settings in `${config_path}/cassandra.yaml`. * On CentOS 7 if the `init` service provider is used, then cassandra is added as a system service. * Optionally ensures that the Cassandra service is enabled and running. * On Debian systems: * Optionally replace ```/etc/init.d/cassandra``` with a workaround for [CASSANDRA-9822](https://issues.apache.org/jira/browse/CASSANDRA-9822). #### What the cassandra::datastax_agent class affects * Optionally installs the DataStax agent. * Optionally sets JAVA_HOME in **/etc/default/datastax-agent**. #### What the cassandra::datastax_repo class affects * Optionally configures a Yum repository to install the Cassandra packages from (on Red Hat). * Optionally configures an Apt repository to install the Cassandra packages from (on Debian). #### What the cassandra::firewall_ports class affects * Optionally configures the firewall for the Cassandra related network ports. #### What the cassandra::java class affects * Optionally installs a JRE/JDK package (e.g. java-1.7.0-openjdk) and the Java Native Access (JNA). #### What the cassandra::optutils class affects * Optionally installs the Cassandra support tools (e.g. cassandra22-tools). ### Upgrading We follow [SemVer Versioning](http://semver.org/) and an update of the major release (i.e. from 1.*Y*.*Z* to 2.*Y*.*Z*) will indicate a significant change to the API which will most probably require a change to your manifest. #### Changes in 2.0.0 This is a major change to the API and you will more than likely need to change your manifest to accomodate these changes. The `service_ensure` attribute of the cassandra class now defaults to *undef*, users who do want to manage service status in Puppet can still set it to true. If leaving the value at the default and setting `service_refresh` and `service_enable` to false will mean that the user and not Puppet running will control the running state of Cassandra. This currently works OK on the Red Hat family, but has issues on Debian due to [CASSANDRA-2356](https://issues.apache.org/jira/browse/CASSANDRA-2356) during an initial install or package upgrade. All the functionality relating to OpsCenter has been divested to the [locp/opscenter](https://forge.puppet.com/locp/opscenter) module on Puppet Forge. It should also be noted that the module no longer creates directories for the `data`, `commitlog`, `saved_caches` and for Cassandra 3 the `hints` directory. These resources will now need to be defined in your manifest/profile. For a list of features that have been deprecated in this release, please see https://github.com/locp/cassandra/wiki/Deprecations For details on migrating from the version 1.X.X attributes to the `settings` hash, see https://github.com/locp/cassandra/wiki/Version-1.X.Y-Template-Defaults-Shown-As-2.X.Y-Hash Please also see the notes for 2.0.0 in the [CHANGELOG](https://forge.puppet.com/locp/cassandra/changelog). #### Changes in 1.19.0 The hints_directory documentation will cause a change in the cassandra.yaml file regardless of the value you set it to. If you do not wish this to result in a refesh of the Cassandra service, please set service_refresh to false. #### Changes in 1.9.2 Now that Cassandra 3 is available from the DataStax repositories, there is a problem (especially on Debian) with the operating system package manager attempting to install Cassandra 3. This can be mitigated against using something similar to the code in this modules acceptance test. Please note that the default Cassandra package name has now been changed from 'dsc'. See the documentation for cassandra::package_name below for details. ```puppet if $::osfamily == 'RedHat' { $version = '2.2.4-1' } else { $version = '2.2.4' } class { 'cassandra': package_ensure => $version, } ``` #### Changes in 1.8.0 A somewhat embarrassing correction to the spelling of the cassandra::fail_on_non_suppoted_os to cassandra::fail_on_non_supported_os. #### Issues when Upgrading to 1.4.0 Unfortunately both releases 1.3.7 and 1.4.0 have subsequently been found to call a refresh service even when no changes had been made to the underlying configuration. In release 1.8.0 (somewhat belatedly) the service_refresh flag has been introduced to mitigate against similar problems. #### Issues When Upgrading to 1.3.7 * Please see the notes for 1.4.0. #### Changes in 1.0.0 * cassandra::cassandra_package_ensure has been renamed to cassandra::package_ensure. * cassandra::cassandra_package_name has been renamed to cassandra::package_name. #### Changes in 0.4.0 There is now a cassandra::datastax_agent class, therefore: * cassandra::datastax_agent_package_ensure has now been replaced with cassandra::datastax_agent::package_ensure. * cassandra::datastax_agent_service_enable has now been replaced with cassandra::datastax_agent::service_enable. * cassandra::datastax_agent_service_ensure has now been replaced with cassandra::datastax_agent::service_ensure. * cassandra::datastax_agent_package_name has now been replaced with cassandra::datastax_agent::package_name. * cassandra::datastax_agent_service_name has now been replaced with cassandra::datastax_agent::service_name. Likewise now there is a new class for handling the installation of Java: * cassandra::java_package_ensure has now been replaced with cassandra::java::ensure. * cassandra::java_package_name has now been replaced with cassandra::java::package_name. Also there is now a class for installing the optional utilities: * cassandra::cassandra_opt_package_ensure has now been replaced with cassandra::optutils:ensure. * cassandra::cassandra_opt_package_name has now been replaced with cassandra::optutils:package_name. #### Changes in 0.3.0 * cassandra_opt_package_ensure changed from 'present' to undef. * The manage_service option has been replaced with service_enable and service_ensure. ### Beginning with Cassandra Create a Cassandra 2.X cluster called MyCassandraCluster which uses the GossipingPropertyFileSnitch and password authentication. In this very basic example the node itself becomes a seed for the cluster and the credentials will default to a user called cassandra with a password called of cassandra.. ```puppet # Cassandra pre-requisites include cassandra::datastax_repo include cassandra::java class { 'cassandra': settings => { 'authenticator' => 'PasswordAuthenticator', 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'listen_address' => $::ipaddress, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => $::ipaddress, }, ], }, ], 'start_native_transport' => true, }, require => Class['cassandra::datastax_repo', 'cassandra::java'], } ``` For this code to run with version 3.X of Cassandra, the `hints_directory` will also need to be specified: ```puppet ... class { 'cassandra': settings => { ... 'hints_directory' => '/var/lib/cassandra/hints', ... }, require => Class['cassandra::datastax_repo', 'cassandra::java'], } ``` +### Hiera + +In your top level node classification (usually `common.yaml`), add the +settings hash and all the tweaks you want all the clusters to use: + +```YAML +cassandra::baseline_settings: + authenticator: AllowAllAuthenticator + authorizer: AllowAllAuthorizer + auto_bootstrap: true + auto_snapshot: true + ... +``` + +Then, in the individual node classification add the parts which define +the cluster: + +```YAML +cassandra::settings: + cluster_name: developer playground cassandra cluster +cassandra::dc: Onsite1 +cassandra::rack: RAC1 +cassandra::package_ensure: 3.0.5-1 +cassandra::package_name: cassandra30 +``` + ## Usage ### Setup a keyspace and users We assume that authentication has been enabled for the cassandra cluster and we are connecting with the default user name and password ('cassandra/cassandra'). In this example, we create a keyspace (mykeyspace) with a table called 'users' and an index called 'users_lname_idx'. We also add three users (to Cassandra, not the mykeyspace.users table) called spillman, akers and boone while ensuring that a user called lucan is absent. ```puppet class { 'cassandra': ... } class { 'cassandra::schema': cqlsh_password => 'cassandra', cqlsh_user => 'cassandra', cqlsh_host => $::ipaddress, indexes => { 'users_lname_idx' => { table => 'users', keys => 'lname', keyspace => 'mykeyspace', }, }, keyspaces => { 'mykeyspace' => { durable_writes => false, replication_map => { keyspace_class => 'SimpleStrategy', replication_factor => 1, }, } }, permissions => { 'Grant select permissions to spillman to all keyspaces' => { permission_name => 'SELECT', user_name => 'spillman', }, 'Grant modify to to keyspace mykeyspace to akers' => { keyspace_name => 'mykeyspace', permission_name => 'MODIFY', user_name => 'akers', }, 'Grant alter permissions to mykeyspace to boone' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', user_name => 'boone', }, 'Grant ALL permissions to mykeyspace.users to gbennet' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', table_name => 'users', user_name => 'gbennet', }, }, tables => { 'users' => { columns => { user_id => 'int', fname => 'text', lname => 'text', 'PRIMARY KEY' => '(user_id)', }, keyspace => 'mykeyspace', }, }, users => { 'spillman' => { password => 'Niner27', }, 'akers' => { password => 'Niner2', superuser => true, }, 'boone' => { password => 'Niner75', }, 'gbennet' => { 'password' => 'foobar', }, 'lucan' => { 'ensure' => absent }, }, } ``` ### Create a Cluster in a Single Data Center In the DataStax documentation _Initializing a multiple node cluster (single data center)_ there is a basic example of a six node cluster with two seeds to be created in a single data center spanning two racks. The nodes in the cluster are: **Node Name** | **IP Address** | ---------------|----------------| node0 (seed 1) | 110.82.155.0 | node1 | 110.82.155.1 | node2 | 110.82.155.2 | node3 (seed 2) | 110.82.156.3 | node4 | 110.82.156.4 | node5 | 110.82.156.5 | Each node is configured to use the GossipingPropertyFileSnitch and 256 virtual nodes (vnodes). The name of the cluster is _MyCassandraCluster_. Also, while building the initial cluster, we are setting the auto_bootstrap to false. In this initial example, we are going to expand the example by: * Ensuring that the software is installed via the DataStax Community repository by including `cassandra::datastax_repo`. This needs to be executed before the Cassandra package is installed. * That a suitable Java Runtime environment (JRE) is installed with Java Native Access (JNA) by including `cassandra::java`. This need to be executed before the Cassandra service is started. ```puppet node /^node\d+$/ { class { 'cassandra::datastax_repo': before => Class['cassandra'] } class { 'cassandra::java': before => Class['cassandra'] } class { 'cassandra': settings => { 'authenticator' => 'AllowAllAuthenticator', 'auto_bootstrap' => false, 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'hints_directory' => '/var/lib/cassandra/hints', 'listen_interface' => 'eth1', 'num_tokens' => 256, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => '110.82.155.0,110.82.156.3', }, ], }, ], 'start_native_transport' => true, }, } } ``` The default value for the num_tokens is already 256, but it is included in the example for clarity. Do not forget to either set auto_bootstrap to true or not set the attribute at all after initializing the cluster. ### Create a Cluster in Multiple Data Centers To continue with the examples provided by DataStax, we look at the example for a cluster across multiple data centers . **Node Name** | **IP Address** | **Data Center** | **Rack** | ---------------|----------------|-----------------|----------| node0 (seed 1) | 10.168.66.41 | DC1 | RAC1 | node1 | 10.176.43.66 | DC1 | RAC1 | node2 | 10.168.247.41 | DC1 | RAC1 | node3 (seed 2) | 10.176.170.59 | DC2 | RAC1 | node4 | 10.169.61.170 | DC2 | RAC1 | node5 | 10.169.30.138 | DC2 | RAC1 | For the sake of simplicity, we will confine this example to the nodes: ```puppet node /^node[012]$/ { class { 'cassandra': dc => 'DC1', settings => { 'authenticator' => 'AllowAllAuthenticator', 'auto_bootstrap' => false, 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'hints_directory' => '/var/lib/cassandra/hints', 'listen_interface' => 'eth1', 'num_tokens' => 256, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => '110.82.155.0,110.82.156.3', }, ], }, ], 'start_native_transport' => true, }, } } node /^node[345]$/ { class { 'cassandra': dc => 'DC2', settings => { 'authenticator' => 'AllowAllAuthenticator', 'auto_bootstrap' => false, 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'hints_directory' => '/var/lib/cassandra/hints', 'listen_interface' => 'eth1', 'num_tokens' => 256, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => '110.82.155.0,110.82.156.3', }, ], }, ], 'start_native_transport' => true, }, } } ``` We don't need to specify the rack name (with the rack attribute) as RAC1 is the default value. Again, do not forget to either set auto_bootstrap to true or not set the attribute at all after initializing the cluster. ## Reference ### Public Classes * [cassandra](http://locp.github.io/cassandra/puppet_classes/cassandra.html) * [cassandra::datastax_agent] (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Adatastax_agent.html) * [cassandra::datastax_repo] (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Adatastax_repo.html) * [cassandra::firewall_ports] (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Afirewall_ports.html) * [cassandra::java] (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Ajava.html) * [cassandra::optutils] (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Aoptutils.html) * [cassandra::schema] (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Aschema.html) +* [cassandra::system::swapoff] + (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Asystem_3A_3Aswapoff.html) +* [cassandra::system::sysctl] + (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Asystem_3A_3Asysctl.html) +* [cassandra::system::transparent_hugepage] + (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Asystem_3A_3Atransparent_hugepage.html) ### Public Defined Types * [cassandra::file] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Afile.html) * [cassandra::schema::cql_type] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Acql_type.html) * [cassandra::schema::index] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Aindex.html) * [cassandra::schema::keyspace] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Akeyspace.html) * [cassandra::schema::permission] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Apermission.html) * [cassandra::schema::table] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Atable.html) * [cassandra::schema::user] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Auser.html) ### Private Defined Types * [cassandra::private::firewall_ports::rule] (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aprivate_3A_3Afirewall_ports_3A_3Arule.html) ### Facts * [cassandracmsheapnewsize] (http://locp.github.io/cassandra/top-level-namespace.html#cassandracmsheapnewsize-instance_method) * [cassandracmsmaxheapsize] (http://locp.github.io/cassandra/top-level-namespace.html#cassandracmsmaxheapsize-instance_method) * [cassandraheapnewsize] (http://locp.github.io/cassandra/top-level-namespace.html#cassandraheapnewsize-instance_method) * [cassandramajorversion] (http://locp.github.io/cassandra/top-level-namespace.html#cassandramajorversion-instance_method) * [cassandramaxheapsize] (http://locp.github.io/cassandra/top-level-namespace.html#cassandramaxheapsize-instance_method) * [cassandraminorversion] (http://locp.github.io/cassandra/top-level-namespace.html#cassandraminorversion-instance_method) * [cassandrapatchversion] (http://locp.github.io/cassandra/top-level-namespace.html#cassandrapatchversion-instance_method) * [cassandrarelease] (http://locp.github.io/cassandra/top-level-namespace.html#cassandrarelease-instance_method) ## Limitations -When using a Ruby version before 1.9.0, the contents of the Cassandra +* When using a Ruby version before 1.9.0, the contents of the Cassandra configuration file may change order of elements due to a problem with to_yaml in earlier versions of Ruby. - -When creating key spaces, indexes, cql_types and users the settings will only +* When creating key spaces, indexes, cql_types and users the settings will only be used to create a new resource if it does not currently exist. If a change is made to the Puppet manifest but the resource already exits, this change will not be reflected. +* At the moment the `cassandra::system::transparent_hugepage` does not +persist between reboots. ## Development Contributions will be gratefully accepted. Please go to the project page, fork the project, make your changes locally and then raise a pull request. Details on how to do this are available at https://guides.github.com/activities/contributing-to-open-source. Please also see the [CONTRIBUTING.md](https://github.com/locp/cassandra/blob/master/CONTRIBUTING.md) page for project specific requirements. ### Additional Contributers For a list of contributers see [CONTRIBUTING.md](https://github.com/locp/cassandra/blob/master/CONTRIBUTING.md) and https://github.com/locp/cassandra/graphs/contributors diff --git a/Vagrantfile b/Vagrantfile index 5cbf5e4..3e43714 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -1,146 +1,51 @@ -# -*- mode: ruby -*- -# vi: set ft=ruby : -require 'json' - -# Vagrantfile API/syntax version. Don't touch unless you know what you're doing! -VAGRANTFILE_API_VERSION = '2'.freeze - -Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| - # All Vagrant configuration is done here. The most common configuration - # options are documented and commented below. For a complete reference, - # please see the online documentation at vagrantup.com. - - # Every Vagrant virtual environment requires a box to build off of. - config.vm.hostname = 'node0' - config.vm.box = 'puppetlabs/centos-7.0-64-nocm' - # config.vm.box = 'puppetlabs/ubuntu-14.04-64-nocm' - config.puppet_install.puppet_version = '3.8.2' - config.vm.synced_folder '.', '/etc/puppet/modules/cassandra' - - # Enable provisioning with Puppet stand alone. Puppet manifests - # are contained in a directory path relative to this Vagrantfile. - # You will need to create the manifests directory and a manifest in - # the file default.pp in the manifests_path directory. - # - # generate list of puppet modules to be installed from metadata.json - metadata_json_file = "#{File.dirname(__FILE__)}/metadata.json" - if File.exist?(metadata_json_file) - JSON.parse(File.read(metadata_json_file))['dependencies'].each do |key, _value| - module_name = key['name'].to_s - config.vm.provision 'shell', - inline: "puppet module install #{module_name}" - end - else - puts 'metadata.json not found; skipping install of dependencies' +# TODO: Find out why cassandra::params not working with vagrant and CentOS 6.2 +# TODO: Find out why cassandra::params not working with vagrant and CentOS 7.0 +# TODO: Find out why cassandra::params not working with vagrant and Ubuntu 12.04. +# TODO: Have vagrant dynamically discover Puppet modules from metadata.json. +Vagrant.configure('2') do |config| + # config.vm.box = 'puppetlabs/centos-6.6-64-puppet' + # config.vm.box = 'puppetlabs/centos-7.0-64-puppet' + config.vm.box = 'puppetlabs/debian-7.8-64-puppet' + # config.vm.box = 'puppetlabs/debian-8.2-64-puppet' + # config.vm.box = "puppetlabs/ubuntu-12.04-64-puppet" + # config.vm.box = "puppetlabs/ubuntu-14.04-64-puppet" + # config.vm.box = "puppetlabs/ubuntu-16.04-64-puppet" + + config.vm.provider 'virtualbox' do |vm| + vm.memory = 2048 + vm.cpus = 2 end - config.vm.provision 'puppet' do |puppet| - puppet.manifests_path = 'examples' - puppet.manifest_file = 'getting_started.pp' + puppet_environment = 'vagrant' + puppet_environment_path_on_guest = "/etc/puppetlabs/code/environments/#{puppet_environment}" + module_path_on_guest = "#{puppet_environment_path_on_guest}/modules" + + config.vm.synced_folder './vagrant', + '/etc/puppetlabs/code/environments/vagrant' + config.vm.synced_folder '.', + '/etc/puppetlabs/code/environments/vagrant/modules/cassandra' + config.vm.synced_folder './spec/acceptance/hieradata', + '/etc/puppetlabs/code/environments/vagrant/hieradata' + + config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/ || mkdir #{puppet_environment_path_on_guest}" + config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/apt || puppet module install puppetlabs-apt --environment=#{puppet_environment}" + config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/firewall || puppet module install puppetlabs-firewall --environment=#{puppet_environment}" + config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/inifile || puppet module install puppetlabs-inifile --environment=#{puppet_environment}" + config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/stdlib || puppet module install puppetlabs-stdlib --environment=#{puppet_environment}" + + config.vm.provision :puppet do |puppet| + puppet.options = ENV['PUPPET_OPTS'].split(' ') if ENV.key?('PUPPET_OPTS') # See http://stackoverflow.com/a/27540417/224334 + puppet.options = '--verbose --debug' if ENV['PUPPET_VERBOSE'] + puppet.hiera_config_path = 'vagrant/hiera.yaml' + puppet.environment = puppet_environment + puppet.environment_path = './' + puppet.manifests_path = "#{puppet_environment}/manifests" + puppet.manifest_file = 'site.pp' + puppet.facter = { + project_name: 'ENGLISH NAME Of PROJECT', # EDIT THIS LINE + } end - # Have something like the following in puppet/manifests/local.pp: - # node default { - # require '::base' - # include '::foobar' - # } - # - # Create a forwarded port mapping which allows access to a specific port - # within the machine from a port on the host machine. In the example below, - # accessing "localhost:8080" will access port 80 on the guest machine. - # config.vm.network "forwarded_port", guest: 80, host: 8080 - config.vm.network 'forwarded_port', guest: 8888, host: 8888 - - # Disable automatic box update checking. If you disable this, then - # boxes will only be checked for updates when the user runs - # `vagrant box outdated`. This is not recommended. - # config.vm.box_check_update = false - - # Create a private network, which allows host-only access to the machine - # using a specific IP. - # config.vm.network "private_network", ip: "192.168.33.10" - - # Create a public network, which generally matched to bridged network. - # Bridged networks make the machine appear as another physical device on - # your network. - # config.vm.network "public_network" - - # If true, then any SSH connections made will enable agent forwarding. - # Default value: false - # config.ssh.forward_agent = true - - # Share an additional folder to the guest VM. The first argument is - # the path on the host to the actual folder. The second argument is - # the path on the guest to mount the folder. And the optional third - # argument is a set of non-required options. - # config.vm.synced_folder "../data", "/vagrant_data" - - # Provider-specific configuration so you can fine-tune various - # backing providers for Vagrant. These expose provider-specific options. - # Example for VirtualBox: - # - config.vm.provider 'virtualbox' do |vb| - # Don't boot with headless mode - # vb.gui = true - - # Use VBoxManage to customize the VM. For example to change memory: - vb.customize ['modifyvm', :id, '--memory', '2048'] - end - # - # View the documentation for the provider you're using for more - # information on available options. - - # Enable provisioning with CFEngine. CFEngine Community packages are - # automatically installed. For example, configure the host as a - # policy server and optionally a policy file to run: - # - # config.vm.provision "cfengine" do |cf| - # cf.am_policy_hub = true - # # cf.run_file = "motd.cf" - # end - # - # You can also configure and bootstrap a client to an existing - # policy server: - # - # config.vm.provision "cfengine" do |cf| - # cf.policy_server_address = "10.0.2.15" - # end - - # Enable provisioning with chef solo, specifying a cookbooks path, roles - # path, and data_bags path (all relative to this Vagrantfile), and adding - # some recipes and/or roles. - # - # config.vm.provision "chef_solo" do |chef| - # chef.cookbooks_path = "../my-recipes/cookbooks" - # chef.roles_path = "../my-recipes/roles" - # chef.data_bags_path = "../my-recipes/data_bags" - # chef.add_recipe "mysql" - # chef.add_role "web" - # - # # You may also specify custom JSON attributes: - # chef.json = { mysql_password: "foo" } - # end - - # Enable provisioning with chef server, specifying the chef server URL, - # and the path to the validation key (relative to this Vagrantfile). - # - # The Opscode Platform uses HTTPS. Substitute your organization for - # ORGNAME in the URL and validation key. - # - # If you have your own Chef Server, use the appropriate URL, which may be - # HTTP instead of HTTPS depending on your configuration. Also change the - # validation key to validation.pem. - # - # config.vm.provision "chef_client" do |chef| - # chef.chef_server_url = "https://api.opscode.com/organizations/ORGNAME" - # chef.validation_key_path = "ORGNAME-validator.pem" - # end - # - # If you're using the Opscode platform, your validator client is - # ORGNAME-validator, replacing ORGNAME with your organization name. - # - # If you have your own Chef Server, the default validation client name is - # chef-validator, unless you changed the configuration. - # - # chef.validation_client_name = "ORGNAME-validator" + config.vm.network :forwarded_port, guest: 22, host: 2223, auto_correct: true, id: 'ssh' + config.vm.network :forwarded_port, guest: 3000, host: 3000 end diff --git a/circle.yml b/circle.yml index 0fb401f..8c4da2e 100644 --- a/circle.yml +++ b/circle.yml @@ -1,44 +1,42 @@ --- branches: ignore: - gh-pages # list of branches to ignore dependencies: cache_directories: - "/opt/circleci/.rvm/gems" override: - bundle install --without development acceptance: timeout: 240 environment: PUPPET_VERSION: '~> 3.8' STRICT_VARIABLES: yes post: - rvm cleanup all - bundle clean --force deployment: master: branch: master commands: - git config --global user.email "info@locp.co.uk" - git config --global user.name "CircleCI" - bundle exec rake strings:gh_pages:update - bundle exec rake deploy machine: environment: PUPPET_VERSION: '~> 3.8' STRICT_VARIABLES: yes ruby: - version: 2.1.6 - services: - - docker + version: 2.1.9 notify: webhooks: - url: "https://webhooks.gitter.im/e/da8c065419a91983f0cf" test: override: - bundle exec puppet --version - bundle exec travis-lint - bundle exec rake metadata_lint - bundle exec rake lint - bundle exec rake validate - bundle exec rake spec SPEC_OPTS="--format documentation --format RspecJunitFormatter -o $CIRCLE_TEST_REPORTS/rspec/puppet.xml" diff --git a/examples/getting_started.pp b/examples/getting_started.pp index 31d61d8..e2698ab 100644 --- a/examples/getting_started.pp +++ b/examples/getting_started.pp @@ -1,191 +1,198 @@ ############################################################################# # This is for placing in the getting started section of the README file. ############################################################################# # Install Cassandra 2.2.5 onto a system and create a basic keyspace, table # and index. The node itself becomes a seed for the cluster. # # Tested on CentOS 7 ############################################################################# # Cassandra pre-requisites -include cassandra::datastax_repo -include cassandra::java +require cassandra::datastax_repo +require cassandra::system::sysctl +require cassandra::system::transparent_hugepage +require cassandra::java + +class { 'cassandra::system::swapoff': + device => '/dev/mapper/centos-swap', + before => Class['cassandra'], +} # Create a cluster called MyCassandraCluster which uses the # GossipingPropertyFileSnitch. In this very basic example # the node itself becomes a seed for the cluster. class { 'cassandra': commitlog_directory => '/var/lib/cassandra/commitlog', data_file_directories => ['/var/lib/cassandra/data'], hints_directory => '/var/lib/cassandra/hints', package_name => 'cassandra30', saved_caches_directory => '/var/lib/cassandra/saved_caches', settings => { 'authenticator' => 'PasswordAuthenticator', 'authorizer' => 'CassandraAuthorizer', 'cluster_name' => 'MyCassandraCluster', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'listen_address' => $::ipaddress, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => $::ipaddress, }, ], }, ], 'start_native_transport' => true, }, service_ensure => running, - require => Class['cassandra::datastax_repo', 'cassandra::java'], + require => Class['cassandra::datastax_repo', 'cassandra::system::sysctl', 'cassandra::java'], } class { 'cassandra::datastax_agent': settings => { 'agent_alias' => { 'setting' => 'agent_alias', 'value' => 'foobar', }, 'stomp_interface' => { 'setting' => 'stomp_interface', 'value' => 'localhost', }, 'async_pool_size' => { 'ensure' => absent, }, }, require => Class['cassandra'], } class { 'cassandra::optutils': package_name => 'cassandra30-tools', require => Class['cassandra'], } class { 'cassandra::schema': cqlsh_password => 'cassandra', cqlsh_user => 'cassandra', cqlsh_host => $::ipaddress, indexes => { 'users_lname_idx' => { table => 'users', keys => 'lname', keyspace => 'mykeyspace', }, }, keyspaces => { 'mykeyspace' => { durable_writes => false, replication_map => { keyspace_class => 'SimpleStrategy', replication_factor => 1, }, }, }, permissions => { 'Grant select permissions to spillman to all keyspaces' => { permission_name => 'SELECT', user_name => 'spillman', }, 'Grant modify to to keyspace mykeyspace to akers' => { keyspace_name => 'mykeyspace', permission_name => 'MODIFY', user_name => 'akers', }, 'Grant alter permissions to mykeyspace to boone' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', user_name => 'boone', }, 'Grant ALL permissions to mykeyspace.users to gbennet' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', table_name => 'users', user_name => 'gbennet', }, }, tables => { 'users' => { columns => { user_id => 'int', fname => 'text', lname => 'text', 'PRIMARY KEY' => '(user_id)', }, keyspace => 'mykeyspace', }, }, users => { 'spillman' => { password => 'Niner27', }, 'akers' => { password => 'Niner2', superuser => true, }, 'boone' => { password => 'Niner75', }, 'gbennet' => { 'password' => 'foobar', }, 'lucan' => { 'ensure' => absent, }, }, } if $::memorysize_mb < 24576.0 { $max_heap_size_in_mb = floor($::memorysize_mb / 2) } elsif $::memorysize_mb < 8192.0 { $max_heap_size_in_mb = floor($::memorysize_mb / 4) } else { $max_heap_size_in_mb = 8192 } $heap_new_size = $::processorcount * 100 cassandra::file { "Set Java/Cassandra max heap size to ${max_heap_size_in_mb}.": file => 'cassandra-env.sh', file_lines => { 'MAX_HEAP_SIZE' => { line => "MAX_HEAP_SIZE='${max_heap_size_in_mb}M'", match => '^#?MAX_HEAP_SIZE=.*', }, }, } cassandra::file { "Set Java/Cassandra heap new size to ${heap_new_size}.": file => 'cassandra-env.sh', file_lines => { 'HEAP_NEWSIZE' => { line => "HEAP_NEWSIZE='${heap_new_size}M'", match => '^#?HEAP_NEWSIZE=.*', }, }, } $tmpdir = '/var/lib/cassandra/tmp' file { $tmpdir: ensure => directory, owner => 'cassandra', group => 'cassandra', require => Package['cassandra'], } cassandra::file { 'Set java.io.tmpdir': file => 'jvm.options', file_lines => { 'java.io.tmpdir' => { line => "-Djava.io.tmpdir=${tmpdir}", }, }, require => File[$tmpdir], } diff --git a/examples/system_settings.pp b/examples/system_settings.pp new file mode 100644 index 0000000..1cdbe3e --- /dev/null +++ b/examples/system_settings.pp @@ -0,0 +1,26 @@ +# Cassandra pre-requisites +require cassandra::datastax_repo +require cassandra::system::transparent_hugepage + +case "${::operatingsystem}-${operatingsystemmajrelease}" { + 'CentOS-6': { + $device = '/dev/mapper/VolGroup-lv_swap' + $sysctl_args = '-e -p' + } + 'CentOS-7': { + $device = '/dev/mapper/centos-swap' + } + default: {} +} + +class { 'cassandra::system::swapoff': + device => $device +} + +if $sysctl_args { + class { 'cassandra::system::sysctl': + sysctl_args => $sysctl_args, + } +} else { + require cassandra::system::sysctl +} diff --git a/manifests/init.pp b/manifests/init.pp index 2f5f24c..91f0cc9 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -1,377 +1,382 @@ # A class for installing the Cassandra package and manipulate settings in the # configuration file. # +# @param baseline_settings [hash] If set, this is a baseline of settings that +# are merged with the `settings` hash. The values of the `settings` +# hash overriding the values in this hash. This is most useful when used +# with hiera. # @param cassandra_2356_sleep_seconds [boolean] # This will provide a workaround for # [CASSANDRA-2356](https://issues.apache.org/jira/browse/CASSANDRA-2356) by # sleeping for the specifed number of seconds after an event involving the # Cassandra package. This option is silently ignored on the Red Hat family # of operating systems as this bug only affects Debian systems. # @param cassandra_9822 [boolean] If set to true, this will apply a patch to the init # file for the Cassandra service as a workaround for # [CASSANDRA-9822](https://issues.apache.org/jira/browse/CASSANDRA-9822). # This this bug only affects Debian systems. # @param cassandra_yaml_tmpl [string] The path to the Puppet template for the # Cassandra configuration file. This allows the user to supply their own # customized template.` # @param commitlog_directory [string] The path to the commitlog directory. # If set, the directory will be managed as a Puppet resource. Do not # specify a value here and in the `settings` hash as they are mutually # exclusive. # @param commitlog_directory_mode [string] The mode for the # `commitlog_directory` is ignored unless `commitlog_directory` is # specified. # @param config_file_mode [string] The permissions mode of the cassandra configuration # file. # @param config_path [string] The path to the cassandra configuration file. # @param data_file_directories [array] The path(s) to the date directory or # directories. # If set, the directories will be managed as a Puppet resource. Do not # specify a value here and in the `settings` hash as they are mutually # exclusive. # @param data_file_directories_mode [string] The mode for the # `data_file_directories` is ignored unless `data_file_directories` is # specified. # @param dc [string] Sets the value for dc in *config_path*/*snitch_properties_file* # http://docs.datastax.com/en/cassandra/2.1/cassandra/architecture/architectureSnitchesAbout_c.html # for more details. # @param dc_suffix [string] Sets the value for dc_suffix in # *config_path*/*snitch_properties_file* see # http://docs.datastax.com/en/cassandra/2.1/cassandra/architecture/architectureSnitchesAbout_c.html # for more details. If the value is *undef* then no change will be made to # the snitch properties file for this setting. # @param fail_on_non_supported_os [boolean] A flag that dictates if the module should # fail if it is not RedHat or Debian. If you set this option to false then # you must also at least set the `config_path` attribute as well. # @param hints_directory [string] The path to the hints directory. # If set, the directory will be managed as a Puppet resource. Do not # specify a value here and in the `settings` hash as they are mutually # exclusive. Do not set this option in Cassandra versions before 3.0.0. # @param hints_directory_mode [string] The mode for the # `hints_directory` is ignored unless `hints_directory` is # specified. # @param package_ensure [present|latest|string] The status of the package specified in # **package_name**. Can be *present*, *latest* or a specific version # number. # @param package_name [string] The name of the Cassandra package which must be available # from a repository. # @param prefer_local [boolean] Sets the value for prefer_local in # *config_path*/*snitch_properties_file* see # http://docs.datastax.com/en/cassandra/2.1/cassandra/architecture/architectureSnitchesAbout_c.html # for more details. Valid values are true, false or *undef*. If the value # is *undef* then change will be made to the snitch properties file for # this setting. # @param rack [string] Sets the value for rack in # *config_path*/*snitch_properties_file* see # http://docs.datastax.com/en/cassandra/2.1/cassandra/architecture/architectureSnitchesAbout_c.html # for more details. # @param rackdc_tmpl [string] The template for creating the snitch properties file. # @param saved_caches_directory [string] The path to the saved caches directory. # If set, the directory will be managed as a Puppet resource. Do not # specify a value here and in the `settings` hash as they are mutually # exclusive. # @param saved_caches_directory_mode [string] The mode for the # `saved_caches_directory` is ignored unless `saved_caches_directory` is # specified. # @param service_enable [boolean] enable the Cassandra service to start at boot time. # @param service_ensure [string] Ensure the Cassandra service is running. Valid values # are running or stopped. # @param service_name [string] The name of the service that runs the Cassandra software. # @param service_provider [string] The name of the provider that runs the service. # If left as *undef* then the OS family specific default will # be used, otherwise the specified value will be used instead. # @param service_refresh [boolean] If set to true, changes to the Cassandra config file # or the data directories will ensure that Cassandra service is refreshed # after the changes. Setting this flag to false will disable this # behaviour, therefore allowing the changes to be made but allow the user # to control when the service is restarted. # @param settings [hash] A hash that is passed to `to_yaml` which dumps the results # to the Cassandra configuring file. The minimum required settings for # Cassandra 2.X are as follows: # # ```puppet # { # 'authenticator' => 'PasswordAuthenticator', # 'cluster_name' => 'MyCassandraCluster', # 'commitlog_directory' => '/var/lib/cassandra/commitlog', # 'commitlog_sync' => 'periodic', # 'commitlog_sync_period_in_ms' => 10000, # 'data_file_directories' => ['/var/lib/cassandra/data'], # 'endpoint_snitch' => 'GossipingPropertyFileSnitch', # 'listen_address' => $::ipaddress, # 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', # 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', # 'seed_provider' => [ # { # 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', # 'parameters' => [ # { # 'seeds' => $::ipaddress, # }, # ], # }, # ], # 'start_native_transport' => true, # } # ``` # For Cassandra 3.X you will also need to specify the `hints_directory` # attribute. # @param snitch_properties_file [string] The name of the snitch properties file. The # full path name would be *config_path*/*snitch_properties_file*. # @param systemctl [string] The full path to the systemctl command. Only # needed when the package is installed. Will silently continue if the # executable does not exist. class cassandra ( + $baseline_settings = {}, $cassandra_2356_sleep_seconds = 5, $cassandra_9822 = false, $cassandra_yaml_tmpl = 'cassandra/cassandra.yaml.erb', $commitlog_directory = undef, $commitlog_directory_mode = '0750', $config_file_mode = '0644', $config_path = $::cassandra::params::config_path, $data_file_directories = undef, $data_file_directories_mode = '0750', $dc = 'DC1', $dc_suffix = undef, $fail_on_non_supported_os = true, $hints_directory = undef, $hints_directory_mode = '0750', $package_ensure = 'present', $package_name = $::cassandra::params::cassandra_pkg, $prefer_local = undef, $rack = 'RAC1', $rackdc_tmpl = 'cassandra/cassandra-rackdc.properties.erb', $saved_caches_directory = undef, $saved_caches_directory_mode = '0750', $service_enable = true, $service_ensure = undef, $service_name = 'cassandra', $service_provider = undef, $service_refresh = true, $settings = {}, $snitch_properties_file = 'cassandra-rackdc.properties', $systemctl = $::cassandra::params::systemctl, ) inherits cassandra::params { if $service_provider != undef { Service { provider => $service_provider, } } $config_file = "${config_path}/cassandra.yaml" $dc_rack_properties_file = "${config_path}/${snitch_properties_file}" case $::osfamily { 'RedHat': { $config_file_require = Package['cassandra'] $config_file_before = [] $config_path_require = Package['cassandra'] $dc_rack_properties_file_require = Package['cassandra'] $dc_rack_properties_file_before = [] $data_dir_require = Package['cassandra'] $data_dir_before = [] if $::operatingsystemmajrelease == 7 and $::cassandra::service_provider == 'init' { exec { "/sbin/chkconfig --add ${service_name}": unless => "/sbin/chkconfig --list ${service_name}", require => Package['cassandra'], before => Service['cassandra'], } } } 'Debian': { $config_file_require = [ User['cassandra'], File[$config_path] ] $config_file_before = Package['cassandra'] $config_path_require = [] $dc_rack_properties_file_require = [ User['cassandra'], File[$config_path] ] $dc_rack_properties_file_before = Package['cassandra'] $data_dir_require = File[$config_file] $data_dir_before = Package['cassandra'] if $cassandra_9822 { file { '/etc/init.d/cassandra': source => 'puppet:///modules/cassandra/CASSANDRA-9822/cassandra', mode => '0555', before => Package['cassandra'], } } # Sleep after package install and before service resource to prevent # possible duplicate processes arising from CASSANDRA-2356. exec { 'CASSANDRA-2356 sleep': command => "/bin/sleep ${cassandra_2356_sleep_seconds}", refreshonly => true, user => 'root', subscribe => Package['cassandra'], before => Service['cassandra'], } group { 'cassandra': ensure => present, } $user = 'cassandra' user { $user: ensure => present, comment => 'Cassandra database,,,', gid => 'cassandra', home => '/var/lib/cassandra', shell => '/bin/false', managehome => true, require => Group['cassandra'], } # End of CASSANDRA-2356 specific resources. } default: { $config_file_before = [] $config_file_require = [ User['cassandra'], File[$config_path] ] $config_path_require = [] $dc_rack_properties_file_require = Package['cassandra'] $dc_rack_properties_file_before = [] if $fail_on_non_supported_os { fail("OS family ${::osfamily} not supported") } else { warning("OS family ${::osfamily} not supported") } } } package { 'cassandra': ensure => $package_ensure, name => $package_name, notify => Exec['cassandra_reload_systemctl'], } exec { 'cassandra_reload_systemctl': command => "${systemctl} daemon-reload", onlyif => "test -x ${systemctl}", path => ['/usr/bin', '/bin'], refreshonly => true, } file { $config_path: ensure => directory, group => 'cassandra', owner => 'cassandra', mode => '0755', require => $config_path_require, } if $commitlog_directory { file { $commitlog_directory: ensure => directory, owner => 'cassandra', group => 'cassandra', mode => $commitlog_directory_mode, require => $data_dir_require, before => $data_dir_before, } $commitlog_directory_settings = merge($settings, { 'commitlog_directory' => $commitlog_directory, }) } else { $commitlog_directory_settings = $settings } if is_array($data_file_directories) { file { $data_file_directories: ensure => directory, owner => 'cassandra', group => 'cassandra', mode => $data_file_directories_mode, require => $data_dir_require, before => $data_dir_before, } $data_file_directories_settings = merge($settings, { 'data_file_directories' => $data_file_directories, }) } else { $data_file_directories_settings = $settings } if $hints_directory { file { $hints_directory: ensure => directory, owner => 'cassandra', group => 'cassandra', mode => $hints_directory_mode, require => $data_dir_require, before => $data_dir_before, } $hints_directory_settings = merge($settings, { 'hints_directory' => $hints_directory, }) } else { $hints_directory_settings = $settings } if $saved_caches_directory { file { $saved_caches_directory: ensure => directory, owner => 'cassandra', group => 'cassandra', mode => $saved_caches_directory_mode, require => $data_dir_require, before => $data_dir_before, } $saved_caches_directory_settings = merge($settings, { 'saved_caches_directory' => $saved_caches_directory, }) } else { $saved_caches_directory_settings = $settings } - $merged_settings = merge($settings, + $merged_settings = merge($baseline_settings, $settings, $commitlog_directory_settings, $data_file_directories_settings, $hints_directory_settings, $saved_caches_directory_settings) file { $config_file: ensure => present, owner => 'cassandra', group => 'cassandra', content => template($cassandra_yaml_tmpl), mode => $config_file_mode, require => $config_file_require, before => $config_file_before, } file { $dc_rack_properties_file: ensure => file, content => template($rackdc_tmpl), owner => 'cassandra', group => 'cassandra', mode => '0644', require => $dc_rack_properties_file_require, before => $dc_rack_properties_file_before, } if $package_ensure != 'absent' and $package_ensure != 'purged' { if $service_refresh { service { 'cassandra': ensure => $service_ensure, name => $service_name, enable => $service_enable, subscribe => [ File[$config_file], File[$dc_rack_properties_file], Package['cassandra'], ], } } else { service { 'cassandra': ensure => $service_ensure, name => $service_name, enable => $service_enable, require => [ File[$config_file], File[$dc_rack_properties_file], Package['cassandra'], ], } } } } diff --git a/manifests/params.pp b/manifests/params.pp index b204681..39cd005 100644 --- a/manifests/params.pp +++ b/manifests/params.pp @@ -1,40 +1,51 @@ -# This class is meant to be called from cassandra. -# It sets variables according to platform -# -# Variables -# --------- -# * `$::cassandra::params::cassandra_pkg` -# defaults to 'cassandra' on Debian and 'cassandra22' on Red Hat. -# * `$::cassandra::params::config_path` -# defaults to '/etc/cassandra' on Debian and '/etc/cassandra/default.conf' on Red Hat. -# * `$::cassandra::params::java_package` -# defaults to 'openjdk-7-jre-headless' on Debian and 'java-1.8.0-openjdk-headless' on Red Hat. -# * `$::cassandra::params::jna_package_name` -# defaults to 'libjna-java' on Debian and 'jna' on Red Hat. -# * `$::cassandra::params::optutils_package_name` -# defaults to 'cassandra-tools' on Debian and 'cassandra22-tools' on Red Hat. -# * `$::cassandra::params::systemctl` -# defaults to '/bin/systemctl' on Debian and '/usr/bin/systemctl' on Red Hat. +# This class is meant to be called from the locp-cassandra module. +# It sets variables according to platform. class cassandra::params { case $::osfamily { 'Debian': { + case $::operatingsystemmajrelease { + 12.04: { + $net_ipv4_tcp_rmem = '4096 87380 16777216' + $net_ipv4_tcp_wmem = '4096 65536 16777216' + } + default: { + $net_ipv4_tcp_rmem = '4096, 87380, 16777216' + $net_ipv4_tcp_wmem = '4096, 65536, 16777216' + } + } + $cassandra_pkg = 'cassandra' $config_path = '/etc/cassandra' $java_package = 'openjdk-7-jre-headless' $jna_package_name = 'libjna-java' $optutils_package_name = 'cassandra-tools' + $sysctl_file = '/etc/sysctl.d/10-cassandra.conf' $systemctl = '/bin/systemctl' } 'RedHat': { + case $::operatingsystemmajrelease { + 6: { + $net_ipv4_tcp_rmem = '4096 87380 16777216' + $net_ipv4_tcp_wmem = '4096 65536 16777216' + $sysctl_file = '/etc/sysctl.conf' + } + 7: { + $net_ipv4_tcp_rmem = '4096, 87380, 16777216' + $net_ipv4_tcp_wmem = '4096, 65536, 16777216' + $sysctl_file = '/etc/sysctl.d/10-cassandra.conf' + } + default: {} + } + $cassandra_pkg = 'cassandra22' $config_path = '/etc/cassandra/default.conf' $java_package = 'java-1.8.0-openjdk-headless' $jna_package_name = 'jna' $optutils_package_name = 'cassandra22-tools' $systemctl = '/usr/bin/systemctl' } default: { $config_path_parents = [] } } } diff --git a/manifests/system/swapoff.pp b/manifests/system/swapoff.pp new file mode 100644 index 0000000..a2244ea --- /dev/null +++ b/manifests/system/swapoff.pp @@ -0,0 +1,27 @@ +# Disable swap on the node as suggested at +# http://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettingsLinux.html +# @param device [string] If provided a mount resource will be created to +# ensure that the device is absent from /etc/fstab to permanently disable swap. +# @param mount [string] The name of the swap mount point. Ignored unless +# `device` has been set. +# @param path [string] The full path to the file to check if swap is enabled. +# @see cassandra::params +class cassandra::system::swapoff( + $device = undef, + $mount = 'swap', + $path = '/proc/swaps', + ) { + exec { 'Disable Swap': + command => 'swapoff --all', + onlyif => "grep -q '^/' ${path}", + path => [ '/bin', '/sbin', '/usr/bin', '/usr/sbin' ], + } + + if $device { + mount { $mount: + ensure => absent, + device => $device, + fstype => 'swap', + } + } +} diff --git a/manifests/system/sysctl.pp b/manifests/system/sysctl.pp new file mode 100644 index 0000000..425d68d --- /dev/null +++ b/manifests/system/sysctl.pp @@ -0,0 +1,113 @@ +# Set Sysctl (kernel runtime parameters) as suggested in +# http://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettingsLinux.html +# +# If any of the values is set into the target file, the sysctl command will +# be called with the provided file name as an argument. +# +# @example Basic requirement +# require cassandra::system::sysctl +# +# @param sysctl_args [string] Passed to the `sysctl` command +# @param sysctl_file [string] Path to the file to insert the settings into. +# @param net_core_optmem_max [integer] The value to set for +# net.core.optmem_max +# @param net_core_rmem_default [integer] The value to set for +# net.core.rmem_default. +# @param net_core_rmem_max [integer] The value to set for net_core_rmem_max. +# @param net_core_wmem_default [integer] The value to set for +# net.core.wmem_default. +# @param net_core_wmem_max [integer] The value to set for net.core.wmem_max. +# @param net_ipv4_tcp_rmem [string] The value to set for net.ipv4.tcp_rmem. +# @param net_ipv4_tcp_wmem [string] The value to set for net.ipv4.tcp_wmem. +# @param vm_max_map_count [integer] The value to set for vm.max_map_count. +# @see cassandra::params +class cassandra::system::sysctl( + $sysctl_args = '-p', + $sysctl_file = $cassandra::params::sysctl_file, + $net_core_optmem_max = 40960, + $net_core_rmem_default = 16777216, + $net_core_rmem_max = 16777216, + $net_core_wmem_default = 16777216, + $net_core_wmem_max = 16777216, + $net_ipv4_tcp_rmem = $::cassandra::params::net_ipv4_tcp_rmem, + $net_ipv4_tcp_wmem = $::cassandra::params::net_ipv4_tcp_wmem, + $vm_max_map_count = 1048575, + ) inherits cassandra::params { + + ini_setting { "net.core.rmem_max = ${net_core_rmem_max}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'net.core.rmem_max', + value => $net_core_rmem_max, + notify => Exec['Apply sysctl changes'], + } + + ini_setting { "net.core.wmem_max = ${net_core_wmem_max}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'net.core.wmem_max', + value => $net_core_wmem_max, + notify => Exec['Apply sysctl changes'], + } + + ini_setting { "net.core.rmem_default = ${net_core_rmem_default}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'net.core.rmem_default', + value => $net_core_rmem_default, + notify => Exec['Apply sysctl changes'], + } + + ini_setting { "net.core.wmem_default = ${net_core_wmem_default}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'net.core.wmem_default', + value => $net_core_wmem_default, + notify => Exec['Apply sysctl changes'], + } + + ini_setting { "net.core.optmem_max = ${net_core_optmem_max}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'net.core.optmem_max', + value => $net_core_optmem_max, + notify => Exec['Apply sysctl changes'], + } + + ini_setting { "net.ipv4.tcp_rmem = ${net_ipv4_tcp_rmem}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'net.ipv4.tcp_rmem', + value => $net_ipv4_tcp_rmem, + notify => Exec['Apply sysctl changes'], + } + + ini_setting { "net.ipv4.tcp_wmem = ${net_ipv4_tcp_wmem}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'net.ipv4.tcp_wmem', + value => $net_ipv4_tcp_wmem, + notify => Exec['Apply sysctl changes'], + } + + ini_setting { "vm.max_map_count = ${vm_max_map_count}": + ensure => present, + path => $sysctl_file, + section => '', + setting => 'vm.max_map_count', + value => $vm_max_map_count, + notify => Exec['Apply sysctl changes'], + } + + exec { 'Apply sysctl changes': + command => "/sbin/sysctl ${sysctl_args} ${sysctl_file}", + refreshonly => true, + } +} diff --git a/manifests/system/transparent_hugepage.pp b/manifests/system/transparent_hugepage.pp new file mode 100644 index 0000000..90b23c5 --- /dev/null +++ b/manifests/system/transparent_hugepage.pp @@ -0,0 +1,14 @@ +# Disable Transparant Huge Pages as suggested in +# http://docs.datastax.com/en/landing_page/doc/landing_page/recommendedSettingsLinux.html. +# @param path [string] The full path to the file for checking/setting +# if Transparent Hugepages is enabled. +# @see cassandra::params +class cassandra::system::transparent_hugepage ( + $path = '/sys/kernel/mm/transparent_hugepage/defrag', + ) inherits cassandra::params { + exec { 'Disable Java Hugepages': + command => "/bin/echo never > ${path}", + path => [ '/bin', '/usr/bin' ], + unless => "grep -q '\\[never\\]' ${path}", + } +} diff --git a/metadata.json b/metadata.json index a1a1682..07f5d6a 100644 --- a/metadata.json +++ b/metadata.json @@ -1,91 +1,91 @@ { "name": "locp-cassandra", - "version": "2.2.1", + "version": "2.3.0", "author": "locp", "summary": "Installs Cassandra, DataStax Agent & OpsCenter on RHEL/Ubuntu/Debian.", "license": "Apache-2.0", "source": "https://github.com/locp/cassandra", "project_page": "https://github.com/locp/cassandra", "issues_url": "https://github.com/locp/cassandra/issues", "tags": [ "cassandra", "cluster", "database", "datastax", "datastax-agent", "nosql" ], "requirements": [ { "name": "pe", "version_requirement": ">= 3.7.0 < 2015.4.0" }, { "name": "puppet", "version_requirement": ">= 3.0.0 < 5.0.0" } ], "dependencies": [ { "name": "puppetlabs-apt", "version_requirement": ">= 2.0.0 < 3.0.0" }, { "name": "puppetlabs-firewall", "version_requirement": ">= 1.0.0 < 2.0.0" }, { "name": "puppetlabs-inifile", "version_requirement": ">= 1.5.0 < 2.0.0" }, { "name": "puppetlabs-stdlib", "version_requirement": ">= 3.0.0 < 5.0.0" } ], "operatingsystem_support": [ { "operatingsystem": "CentOS", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "OracleLinux", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "RedHat", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "Debian", "operatingsystemrelease": [ "7", "8" ] }, { "operatingsystem": "Scientific", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "Ubuntu", "operatingsystemrelease": [ "12.04", "14.04", "16.04" ] } ] } diff --git a/rake/rake_tasks.rb b/rake/rake_tasks.rb index 11e360f..85a0300 100644 --- a/rake/rake_tasks.rb +++ b/rake/rake_tasks.rb @@ -1,48 +1,48 @@ ############################################################################# # Some module specific rake tasks. ############################################################################# +require 'fileutils' require_relative 'tasks/deploy' desc '[CI Only] Run beaker, but only for pull requests or for release branches.' task :acceptance do - skip = true - travis_branch = ENV['TRAVIS_BRANCH'] - travis_event_type = ENV['TRAVIS_EVENT_TYPE'] + travis_pull_request = ENV['TRAVIS_PULL_REQUEST'] - if travis_event_type == 'pull_request' - skip = false - elsif travis_event_type == 'push' - skip = false if travis_branch =~ /^release-/ || travis_branch =~ /^hotfix-/ - end - - if skip + if travis_pull_request.nil? || (travis_pull_request == 'false') puts 'Skipping acceptance tests.' exit(0) else Rake::Task['beaker'].invoke end end desc '[CI Only] Tag, build and push the module to PuppetForge.' task :deploy do abort('Only deploy from master.') unless ENV['CIRCLE_BRANCH'] == 'master' # Find out what the local version of the module is. file = File.read('metadata.json') data_hash = JSON.parse(file) local_version = data_hash['version'] abort('Unable to find local module version.') unless local_version puts "Module version (local): #{local_version}" Rake::Task['deploy:tag'].invoke(local_version) Rake::Task['deploy:forge'].invoke(local_version) end desc 'Run metadata_lint, rubocop, lint, validate and spec.' task test: [ :metadata_lint, :rubocop, :lint, :validate, :spec ] + +desc 'Clean up after a vagrant run.' +task :vagrant_clean do + module_root = File.expand_path(File.join(__FILE__, '..', '..')) + directory = File.expand_path(File.join(module_root, 'vagrant', 'modules')) + FileUtils.rm_r directory if File.directory?(directory) +end diff --git a/spec/acceptance/bootstrap_spec.rb b/spec/acceptance/bootstrap_spec.rb index 4c59fc4..f9d9a73 100644 --- a/spec/acceptance/bootstrap_spec.rb +++ b/spec/acceptance/bootstrap_spec.rb @@ -1,68 +1,33 @@ require 'spec_helper_acceptance' describe 'Bootstrap' do - whoami_pp = <<-EOS - notify { "operatingsystem: ${::operatingsystem}": } -> - notify { "operatingsystemmajrelease: ${::operatingsystemmajrelease}": } -> - notify { "cassandramaxheapsize: ${::cassandramaxheapsize}": } -> - notify { "cassandracmsmaxheapsize: ${::cassandracmsmaxheapsize}": } -> - notify { "cassandraheapnewsize: ${::cassandraheapnewsize}": } -> - notify { "cassandracmsheapnewsize: ${::cassandracmsheapnewsize}": } - EOS - - describe '########### Identify the node.' do - it 'should work with no errors' do - apply_manifest(whoami_pp, catch_failures: true) - end - end - bootstrap_pp = <<-EOS case downcase($::operatingsystem) { 'centos': { if $::operatingsystemmajrelease == 6 { exec { '/bin/cp /opt/rh/ruby200/enable /etc/profile.d/ruby.sh': } -> exec { '/bin/rm /usr/bin/ruby /usr/bin/gem': } -> exec { '/usr/sbin/alternatives --install /usr/bin/ruby ruby /opt/rh/ruby200/root/usr/bin/ruby 1000': } -> - exec { '/usr/sbin/alternatives --install /usr/bin/gem gem /opt/rh/ruby200/root/usr/bin/gem 1000': } -> - exec { '/bin/cp /opt/rh/python27/enable /etc/profile.d/python.sh': } + exec { '/usr/sbin/alternatives --install /usr/bin/gem gem /opt/rh/ruby200/root/usr/bin/gem 1000': } } } 'ubuntu': { if $::operatingsystemmajrelease == 12.04 { package {'python-software-properties':} -> exec {'/usr/bin/apt-add-repository ppa:brightbox/ruby-ng':} -> exec {'/usr/bin/apt-get update': } -> package {'ruby2.0': } -> exec { '/bin/rm /usr/bin/ruby': } -> exec { '/usr/sbin/update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby2.0 1000': } } - elsif $::operatingsystemmajrelease == 16.04 { - package { 'locales-all': } -> - package { 'net-tools': } -> - package { 'sudo': } -> - package { 'ufw': } -> - package { 'wget': } -> - package { 'ntp': } -> - package { 'python-pip': } -> - package { 'python-minimal': } -> - exec { '/bin/rm -f /usr/sbin/policy-rc.d': } -> - exec { '/usr/bin/wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb': - cwd => '/var/tmp', - creates => '/var/tmp/python-support_1.0.15_all.deb', - } ~> - exec { '/usr/bin/dpkg -i /var/tmp/python-support_1.0.15_all.deb': } -> - package { 'cassandra-driver': - provider => 'pip', - } - } } } EOS describe '########### Node specific manifest.' do it 'should work with no errors' do apply_manifest(bootstrap_pp, catch_failures: true) shell('[ -d /opt/rh/ruby200 ] && /usr/bin/gem install puppet -v 3.8.7 --no-rdoc --no-ri; true') end end end diff --git a/spec/acceptance/cassandra2_spec.rb b/spec/acceptance/cassandra2_spec.rb index 69e246c..99ac7e8 100644 --- a/spec/acceptance/cassandra2_spec.rb +++ b/spec/acceptance/cassandra2_spec.rb @@ -1,453 +1,423 @@ require 'spec_helper_acceptance' describe 'cassandra2', unless: CASSANDRA2_UNSUPPORTED_PLATFORMS.include?(fact('lsbdistrelease')) do cassandra_install_pp = <<-EOS include cassandra::datastax_repo include cassandra::java + # require cassandra::system::sysctl + require cassandra::system::swapoff + require cassandra::system::transparent_hugepage $version = '2.2.8' if $::osfamily == 'RedHat' { $package_ensure = "${version}-1" $cassandra_optutils_package = 'cassandra22-tools' $cassandra_package = 'cassandra22' } else { $package_ensure = $version $cassandra_optutils_package = 'cassandra-tools' $cassandra_package = 'cassandra' exec { '/bin/chown root:root /etc/apt/sources.list.d/datastax.list': unless => '/usr/bin/test -O /etc/apt/sources.list.d/datastax.list', require => Class['cassandra::datastax_agent'] } } - $settings = { - 'authenticator' => 'PasswordAuthenticator', - 'authorizer' => 'CassandraAuthorizer', - 'cluster_name' => 'MyCassandraCluster', - 'commitlog_directory' => '/var/lib/cassandra/commitlog', - 'commitlog_sync' => 'periodic', - 'commitlog_sync_period_in_ms' => 10000, - 'data_file_directories' => ['/var/lib/cassandra/data'], - 'endpoint_snitch' => 'GossipingPropertyFileSnitch', - 'listen_address' => $::ipaddress, - 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', - 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', - 'seed_provider' => [ - { - 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', - 'parameters' => [ - { - 'seeds' => $::ipaddress, - }, - ], - }, - ], - 'start_native_transport' => true, - } - class { 'cassandra': package_ensure => $package_ensure, package_name => $cassandra_package, - settings => $settings, require => Class['cassandra::datastax_repo', 'cassandra::java'] } class { 'cassandra::optutils': package_ensure => $package_ensure, package_name => $cassandra_optutils_package, require => Class['cassandra'] } class { 'cassandra::datastax_agent': require => Class['cassandra'] } # This really sucks but Docker, CentOS 6 and iptables don't play nicely # together. Therefore we can't test the firewall on this platform :-( if $::operatingsystem != CentOS and $::operatingsystemmajrelease != 6 { include '::cassandra::firewall_ports' } EOS describe '########### Cassandra installation.' do it 'should work with no errors' do apply_manifest(cassandra_install_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(cassandra_install_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_create_pp = <<-EOS #{cassandra_install_pp} $cql_types = { 'fullname' => { 'keyspace' => 'mykeyspace', 'fields' => { 'fname' => 'text', 'lname' => 'text', }, }, } $keyspaces = { 'mykeyspace' => { ensure => present, replication_map => { keyspace_class => 'SimpleStrategy', replication_factor => 1, }, durable_writes => false, }, } class { 'cassandra::schema': cql_types => $cql_types, - cqlsh_host => $::ipaddress, cqlsh_password => 'cassandra', cqlsh_user => 'cassandra', indexes => { 'users_lname_idx' => { keyspace => 'mykeyspace', table => 'users', keys => 'lname', }, }, keyspaces => $keyspaces, tables => { 'users' => { 'keyspace' => 'mykeyspace', 'columns' => { 'userid' => 'int', 'fname' => 'text', 'lname' => 'text', 'PRIMARY KEY' => '(userid)', }, }, }, permissions => { 'Grant select permissions to spillman to all keyspaces' => { permission_name => 'SELECT', user_name => 'spillman', }, 'Grant modify to to keyspace mykeyspace to akers' => { keyspace_name => 'mykeyspace', permission_name => 'MODIFY', user_name => 'akers', }, 'Grant alter permissions to mykeyspace to boone' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', user_name => 'boone', }, 'Grant ALL permissions to mykeyspace.users to gbennet' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', table_name => 'users', user_name => 'gbennet', }, }, users => { 'akers' => { password => 'Niner2', superuser => true, }, 'boone' => { password => 'Niner75', }, 'gbennet' => { password => 'Strewth', }, 'spillman' => { password => 'Niner27', }, }, } EOS describe '########### Schema create.' do it 'should work with no errors' do apply_manifest(schema_testing_create_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(schema_testing_create_pp, catch_failures: true).exit_code).to be_zero end end schema_drop_type_pp = <<-EOS #{cassandra_install_pp} $cql_types = { 'fullname' => { 'keyspace' => 'mykeyspace', 'ensure' => 'absent' } } class { 'cassandra::schema': cql_types => $cql_types, - cqlsh_host => $::ipaddress, cqlsh_user => 'akers', cqlsh_password => 'Niner2', } EOS describe '########### Schema drop type.' do it 'should work with no errors' do apply_manifest(schema_drop_type_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(schema_drop_type_pp, catch_failures: true).exit_code).to be_zero end end permissions_revoke_pp = <<-EOS #{cassandra_install_pp} class { 'cassandra::schema': cqlsh_password => 'Niner2', - cqlsh_host => $::ipaddress, cqlsh_user => 'akers', cqlsh_client_config => '/root/.puppetcqlshrc', permissions => { 'Revoke select permissions to spillman to all keyspaces' => { ensure => absent, permission_name => 'SELECT', user_name => 'spillman', }, 'Revoke modify to to keyspace mykeyspace to akers' => { ensure => absent, keyspace_name => 'mykeyspace', permission_name => 'MODIFY', user_name => 'akers', }, 'Revoke alter permissions to mykeyspace to boone' => { ensure => absent, keyspace_name => 'mykeyspace', permission_name => 'ALTER', user_name => 'boone', }, 'Revoke ALL permissions to mykeyspace.users to gbennet' => { ensure => absent, keyspace_name => 'mykeyspace', permission_name => 'ALTER', table_name => 'users', user_name => 'gbennet', }, }, } EOS describe '########### Revoke permissions.' do it 'should work with no errors' do apply_manifest(permissions_revoke_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(permissions_revoke_pp, catch_failures: true).exit_code).to be_zero end end schema_drop_user_pp = <<-EOS #{cassandra_install_pp} class { 'cassandra::schema': cqlsh_password => 'Niner2', - cqlsh_host => $::ipaddress, cqlsh_user => 'akers', cqlsh_client_config => '/root/.puppetcqlshrc', users => { 'boone' => { ensure => absent, }, }, } EOS describe '########### Drop the boone user.' do it 'should work with no errors' do apply_manifest(schema_drop_user_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(schema_drop_user_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_index_pp = <<-EOS #{cassandra_install_pp} class { 'cassandra::schema': - cqlsh_host => $::ipaddress, cqlsh_user => 'akers', cqlsh_password => 'Niner2', indexes => { 'users_lname_idx' => { ensure => absent, keyspace => 'mykeyspace', table => 'users', }, }, } EOS describe '########### Schema drop index.' do it 'should work with no errors' do apply_manifest(schema_testing_drop_index_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(schema_testing_drop_index_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_pp = <<-EOS #{cassandra_install_pp} class { 'cassandra::schema': - cqlsh_host => $ipaddress, cqlsh_password => 'Niner2', cqlsh_user => 'akers', tables => { 'users' => { ensure => absent, keyspace => 'mykeyspace', }, }, } EOS describe '########### Schema drop (table).' do it 'should work with no errors' do apply_manifest(schema_testing_drop_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(schema_testing_drop_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_pp = <<-EOS #{cassandra_install_pp} $keyspaces = { 'mykeyspace' => { ensure => absent, } } class { 'cassandra::schema': - cqlsh_host => $::ipaddress, cqlsh_password => 'Niner2', cqlsh_user => 'akers', keyspaces => $keyspaces, } EOS describe '########### Schema drop (Keyspaces).' do it 'should work with no errors' do apply_manifest(schema_testing_drop_pp, catch_failures: true) end it 'check code is idempotent' do expect(apply_manifest(schema_testing_drop_pp, catch_failures: true).exit_code).to be_zero end end describe service('cassandra') do it do is_expected.to be_running is_expected.to be_enabled end end describe service('datastax-agent') do it do is_expected.to be_running is_expected.to be_enabled end end facts_testing_pp = <<-EOS #{cassandra_install_pp} if $::cassandrarelease != $version { fail("Test1: ${version} != ${::cassandrarelease}") } $assembled_version = "${::cassandramajorversion}.${::cassandraminorversion}.${::cassandrapatchversion}" if $version != $assembled_version { fail("Test2: ${version} != ${::assembled_version}") } if $::cassandramaxheapsize <= 0 { fail('cassandramaxheapsize is not set.') } if $::cassandracmsmaxheapsize <= 0 { fail('cassandracmsmaxheapsize is not set.') } if $::cassandraheapnewsize <= 0 { fail('cassandraheapnewsize is not set.') } if $::cassandracmsheapnewsize <= 0 { fail('cassandracmsheapnewsize is not set.') } EOS describe '########### Facts Tests.' do it 'should work with no errors' do apply_manifest(facts_testing_pp, catch_failures: true) end end describe '########### Gather service information (when in debug mode).' do it 'Show the cassandra system log.' do shell("grep -v -e '^INFO' -e '^\s*INFO' /var/log/cassandra/system.log") end end cassandra_uninstall_pp = <<-EOS Exec { path => [ '/usr/local/bin', '/opt/local/bin', '/usr/bin', '/usr/sbin', '/bin', '/sbin'], logoutput => true, } if $::osfamily == 'RedHat' { $cassandra_optutils_package = 'cassandra22-tools' $cassandra_package = 'cassandra22' } else { $cassandra_optutils_package = 'cassandra-tools' $cassandra_package = 'cassandra' } service { 'cassandra': ensure => stopped, } -> package { $cassandra_optutils_package: - ensure => absent + ensure => purged, } -> package { $cassandra_package: - ensure => absent + ensure => purged, } -> exec { 'rm -rf /var/lib/cassandra/*/* /var/log/cassandra/*': } EOS describe '########### Uninstall Cassandra 2.' do it 'should work with no errors' do apply_manifest(cassandra_uninstall_pp, catch_failures: true) end end end diff --git a/spec/acceptance/cassandra3_spec.rb b/spec/acceptance/cassandra3_spec.rb index c963e7d..a9e16a6 100644 --- a/spec/acceptance/cassandra3_spec.rb +++ b/spec/acceptance/cassandra3_spec.rb @@ -1,399 +1,318 @@ require 'spec_helper_acceptance' describe 'cassandra3' do version = '3.0.9' - lsbdistid = fact('lsbdistid') - lsbmajdistrelease = fact('lsbmajdistrelease') - osdisplay = "#{lsbdistid}#{lsbmajdistrelease}" - - legacy_yml_dump = if (osdisplay == 'CentOS6') || (osdisplay == 'Ubuntu1204') - true - else - false - end cassandra_install_pp = <<-EOS include cassandra::datastax_repo include cassandra::java $version = '#{version}' if $::osfamily == 'RedHat' { $package_ensure = "${version}-1" $cassandra_optutils_package = 'cassandra30-tools' $cassandra_package = 'cassandra30' } else { $cassandra_optutils_package = 'cassandra-tools' $cassandra_package = 'cassandra' $package_ensure = $version if $::lsbdistid == 'Ubuntu' { if $::operatingsystemmajrelease >= 16 { # Workarounds for amonst other things CASSANDRA-11850 Exec { environment => [ 'CQLSH_NO_BUNDLED=TRUE' ] } } } exec { '/bin/chown root:root /etc/apt/sources.list.d/datastax.list': unless => '/usr/bin/test -O /etc/apt/sources.list.d/datastax.list', require => Class['cassandra::datastax_agent'] } } - $settings = { - 'authenticator' => 'PasswordAuthenticator', - 'cluster_name' => 'MyCassandraCluster', - 'commitlog_directory' => '/var/lib/cassandra/commitlog', - 'commitlog_sync' => 'periodic', - 'commitlog_sync_period_in_ms' => 10000, - 'data_file_directories' => ['/var/lib/cassandra/data'], - 'endpoint_snitch' => 'GossipingPropertyFileSnitch', - 'hints_directory' => '/var/lib/cassandra/hints', - 'listen_address' => $::ipaddress, - 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', - 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', - 'seed_provider' => [ - { - 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', - 'parameters' => [ - { - 'seeds' => $::ipaddress, - }, - ], - }, - ], - 'start_native_transport' => true, - } - if versioncmp($::rubyversion, '1.9.0') < 0 { $service_refresh = false } else { $service_refresh = true } class { 'cassandra': + hints_directory => '/var/lib/cassandra/hints', package_ensure => $package_ensure, package_name => $cassandra_package, service_refresh => $service_refresh, - settings => $settings, require => Class['cassandra::datastax_repo', 'cassandra::java'] } class { 'cassandra::optutils': package_ensure => $package_ensure, package_name => $cassandra_optutils_package, require => Class['cassandra'] } class { 'cassandra::datastax_agent': require => Class['cassandra'] } # This really sucks but Docker, CentOS 6 and iptables don't play nicely # together. Therefore we can't test the firewall on this platform :-( if $::operatingsystem != CentOS and $::operatingsystemmajrelease != 6 { include '::cassandra::firewall_ports' } EOS - describe "########### Cassandra #{version} installation on #{osdisplay}" do + describe "########### Cassandra #{version} installation" do it 'should work with no errors' do apply_manifest(cassandra_install_pp, catch_failures: true) end - if legacy_yml_dump - it 'should work with no errors (subsequent run)' do - apply_manifest(cassandra_install_pp, catch_failures: true) - end - else - it 'check code is idempotent' do - expect(apply_manifest(cassandra_install_pp, - catch_failures: true).exit_code).to be_zero - end + it 'check code is idempotent' do + expect(apply_manifest(cassandra_install_pp, + catch_failures: true).exit_code).to be_zero end end schema_testing_create_pp = <<-EOS #{cassandra_install_pp} $cql_types = { 'fullname' => { 'keyspace' => 'mykeyspace', 'fields' => { 'fname' => 'text', 'lname' => 'text', }, }, } $keyspaces = { 'mykeyspace' => { ensure => present, replication_map => { keyspace_class => 'SimpleStrategy', replication_factor => 1, }, durable_writes => false, }, } class { 'cassandra::schema': cql_types => $cql_types, - cqlsh_host => $::ipaddress, cqlsh_password => 'cassandra', cqlsh_user => 'cassandra', indexes => { 'users_lname_idx' => { keyspace => 'mykeyspace', table => 'users', keys => 'lname', }, }, keyspaces => $keyspaces, tables => { 'users' => { 'keyspace' => 'mykeyspace', 'columns' => { 'userid' => 'int', 'fname' => 'text', 'lname' => 'text', 'PRIMARY KEY' => '(userid)', }, }, }, users => { 'spillman' => { password => 'Niner27', }, 'akers' => { password => 'Niner2', superuser => true, }, 'boone' => { password => 'Niner75', }, }, } EOS - describe "########### Schema create #{version} on #{osdisplay}." do + describe "########### Schema create #{version}" do it 'should work with no errors' do apply_manifest(schema_testing_create_pp, catch_failures: true) end - if legacy_yml_dump - it 'should work with no errors (subsequent run)' do - apply_manifest(schema_testing_create_pp, catch_failures: true) - end - else - it 'check code is idempotent' do - expect(apply_manifest(schema_testing_create_pp, catch_failures: true).exit_code).to be_zero - end + it 'check code is idempotent' do + expect(apply_manifest(schema_testing_create_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_type_pp = <<-EOS #{cassandra_install_pp} $cql_types = { 'fullname' => { 'keyspace' => 'mykeyspace', 'ensure' => 'absent' } } class { 'cassandra::schema': cql_types => $cql_types, - cqlsh_host => $::ipaddress, cqlsh_user => 'akers', cqlsh_password => 'Niner2', } EOS - describe "########### Schema drop type #{version} on #{osdisplay}." do + describe "########### Schema drop type #{version}" do it 'should work with no errors' do apply_manifest(schema_testing_drop_type_pp, catch_failures: true) end - if legacy_yml_dump - it 'should work with no errors (subsequent run)' do - apply_manifest(schema_testing_drop_type_pp, catch_failures: true) - end - else - it 'check code is idempotent' do - expect(apply_manifest(schema_testing_drop_type_pp, catch_failures: true).exit_code).to be_zero - end + it 'check code is idempotent' do + expect(apply_manifest(schema_testing_drop_type_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_user_pp = <<-EOS #{cassandra_install_pp} class { 'cassandra::schema': cqlsh_password => 'Niner2', - cqlsh_host => $::ipaddress, cqlsh_user => 'akers', cqlsh_client_config => '/root/.puppetcqlshrc', users => { 'boone' => { ensure => absent, }, }, } EOS - describe "########### Drop the boone user #{version} on #{osdisplay}." do + describe "########### Drop the boone user #{version}" do it 'should work with no errors' do apply_manifest(schema_testing_drop_user_pp, catch_failures: true) end - if legacy_yml_dump - it 'should work with no errors (subsequent run)' do - apply_manifest(schema_testing_drop_user_pp, catch_failures: true) - end - else - it 'check code is idempotent' do - expect(apply_manifest(schema_testing_drop_user_pp, catch_failures: true).exit_code).to be_zero - end + it 'check code is idempotent' do + expect(apply_manifest(schema_testing_drop_user_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_index_pp = <<-EOS #{cassandra_install_pp} class { 'cassandra::schema': - cqlsh_host => $::ipaddress, cqlsh_user => 'akers', cqlsh_password => 'Niner2', indexes => { 'users_lname_idx' => { ensure => absent, keyspace => 'mykeyspace', table => 'users', }, }, } EOS - describe "########### Schema drop index #{version} on #{osdisplay}." do + describe "########### Schema drop index #{version}" do it 'should work with no errors' do apply_manifest(schema_testing_drop_index_pp, catch_failures: true) end - if legacy_yml_dump - it 'should run with no errors (subsequent run)' do - apply_manifest(schema_testing_drop_index_pp, catch_failures: true) - end - else - it 'check code is idempotent' do - expect(apply_manifest(schema_testing_drop_index_pp, catch_failures: true).exit_code).to be_zero - end + it 'check code is idempotent' do + expect(apply_manifest(schema_testing_drop_index_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_pp = <<-EOS #{cassandra_install_pp} class { 'cassandra::schema': - cqlsh_host => $ipaddress, cqlsh_password => 'Niner2', cqlsh_user => 'akers', tables => { 'users' => { ensure => absent, keyspace => 'mykeyspace', }, }, } EOS - describe "########### Schema drop (table) #{version} on #{osdisplay}." do + describe "########### Schema drop (table) #{version}" do it 'should work with no errors' do apply_manifest(schema_testing_drop_pp, catch_failures: true) end - if legacy_yml_dump - it 'should work with no errors (subsequent run)' do - apply_manifest(schema_testing_drop_pp, catch_failures: true) - end - else - it 'check code is idempotent' do - expect(apply_manifest(schema_testing_drop_pp, catch_failures: true).exit_code).to be_zero - end + it 'check code is idempotent' do + expect(apply_manifest(schema_testing_drop_pp, catch_failures: true).exit_code).to be_zero end end schema_testing_drop_pp = <<-EOS #{cassandra_install_pp} $keyspaces = { 'mykeyspace' => { ensure => absent, } } class { 'cassandra::schema': - cqlsh_host => $::ipaddress, cqlsh_password => 'Niner2', cqlsh_user => 'akers', keyspaces => $keyspaces, } EOS - describe "########### Schema drop (Keyspaces) #{version} on #{osdisplay}." do + describe "########### Schema drop (Keyspaces) #{version}" do it 'should work with no errors' do apply_manifest(schema_testing_drop_pp, catch_failures: true) end - if legacy_yml_dump - it 'should work with no errors (subsequent run)' do - apply_manifest(schema_testing_drop_pp, catch_failures: true) - end - else - it 'check code is idempotent' do - expect(apply_manifest(schema_testing_drop_pp, catch_failures: true).exit_code).to be_zero - end + + it 'check code is idempotent' do + expect(apply_manifest(schema_testing_drop_pp, catch_failures: true).exit_code).to be_zero end end describe service('cassandra') do it do is_expected.to be_running is_expected.to be_enabled end end describe service('datastax-agent') do it do is_expected.to be_running is_expected.to be_enabled end end facts_testing_pp = <<-EOS #{cassandra_install_pp} if $::cassandrarelease != $version { fail("Test1: ${version} != ${::cassandrarelease}") } $assembled_version = "${::cassandramajorversion}.${::cassandraminorversion}.${::cassandrapatchversion}" if $version != $assembled_version { fail("Test2: ${version} != ${::assembled_version}") } EOS - describe "########### Facts Tests #{version} on #{osdisplay}." do + describe "########### Facts Tests #{version}" do it 'should work with no errors' do apply_manifest(facts_testing_pp, catch_failures: true) end end describe '########### Gather service information (when in debug mode).' do it 'Show the cassandra system log.' do shell("grep -v -e '^INFO' -e '^\s*INFO' /var/log/cassandra/system.log") end end end diff --git a/spec/acceptance/hieradata/environments/production/data/common.yaml b/spec/acceptance/hieradata/environments/production/data/common.yaml new file mode 100644 index 0000000..f94b99b --- /dev/null +++ b/spec/acceptance/hieradata/environments/production/data/common.yaml @@ -0,0 +1,93 @@ +--- +############################################################################# +# common.yaml +# =========== +# +# A file for hiera data that is relevant to all test nodes. +############################################################################# + +cassandra::baseline_settings: + authenticator: AllowAllAuthenticator + authorizer: AllowAllAuthorizer + auto_snapshot: true + batch_size_warn_threshold_in_kb: 5 + batchlog_replay_throttle_in_kb: 1024 + cas_contention_timeout_in_ms: 1000 + client_encryption_options: + enabled: false + keystore: conf/.keystore + keystore_password: cassandra + cluster_name: "Test Cluster" + column_index_size_in_kb: 64 + commit_failure_policy: stop + commitlog_directory: /var/lib/cassandra/commitlog + commitlog_segment_size_in_mb: 32 + commitlog_sync: periodic + commitlog_sync_period_in_ms: 10000 + compaction_throughput_mb_per_sec: 16 + concurrent_counter_writes: 32 + concurrent_reads: 32 + concurrent_writes: 32 + counter_cache_save_period: 7200 + counter_write_request_timeout_in_ms: 5000 + cross_node_timeout: false + data_file_directories: + - /var/lib/cassandra/data + disk_failure_policy: stop + dynamic_snitch_badness_threshold: 0.1 + dynamic_snitch_reset_interval_in_ms: 600000 + dynamic_snitch_update_interval_in_ms: 100 + endpoint_snitch: SimpleSnitch + hinted_handoff_enabled: true + hinted_handoff_throttle_in_kb: 1024 + incremental_backups: false + index_summary_resize_interval_in_minutes: 60 + inter_dc_tcp_nodelay: false + internode_compression: all + key_cache_save_period: 14400 + listen_address: localhost + max_hint_window_in_ms: 10800000 + max_hints_delivery_threads: 2 + memtable_allocation_type: heap_buffers + native_transport_port: 9042 + num_tokens: 256 + partitioner: org.apache.cassandra.dht.Murmur3Partitioner + permissions_validity_in_ms: 2000 + range_request_timeout_in_ms: 10000 + read_request_timeout_in_ms: 5000 + request_scheduler: org.apache.cassandra.scheduler.NoScheduler + request_timeout_in_ms: 10000 + row_cache_save_period: 0 + row_cache_size_in_mb: 0 + rpc_address: localhost + rpc_keepalive: true + rpc_port: 9160 + rpc_server_type: sync + saved_caches_directory: /var/lib/cassandra/saved_caches + seed_provider: + - class_name: org.apache.cassandra.locator.SimpleSeedProvider + parameters: + - seeds: "127.0.0.1" + server_encryption_options: + internode_encryption: none + keystore: conf/.keystore + keystore_password: cassandra + truststore: conf/.truststore + truststore_password: cassandra + snapshot_before_compaction: false + ssl_storage_port: 7001 + sstable_preemptive_open_interval_in_mb: 50 + start_native_transport: true + start_rpc: true + storage_port: 7000 + thrift_framed_transport_size_in_mb: 15 + tombstone_failure_threshold: 100000 + tombstone_warn_threshold: 1000 + trickle_fsync: false + trickle_fsync_interval_in_kb: 10240 + truncate_request_timeout_in_ms: 60000 + write_request_timeout_in_ms: 2000 +cassandra::cassandra_9822: true +cassandra::dc: LON +cassandra::rack: R101 +cassandra::service_ensure: running diff --git a/spec/acceptance/hieradata/environments/production/data/modules/cassandra.yaml b/spec/acceptance/hieradata/environments/production/data/modules/cassandra.yaml new file mode 100644 index 0000000..ee3c4b8 --- /dev/null +++ b/spec/acceptance/hieradata/environments/production/data/modules/cassandra.yaml @@ -0,0 +1,6 @@ +--- +cassandra::settings: + authenticator: PasswordAuthenticator + authorizer: CassandraAuthorizer + cluster_name: BEAKER + endpoint_snitch: GossipingPropertyFileSnitch diff --git a/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/12.04.yaml b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/12.04.yaml new file mode 100644 index 0000000..70bf92f --- /dev/null +++ b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/12.04.yaml @@ -0,0 +1,13 @@ +--- +cassandra::java::aptkey: + OpenJDK: + id: 'DA1A4A13543B466853BAF164EB9B1D8886F44E2A' + server: 'keyserver.ubuntu.com' + +cassandra::java::aptsource: + OpenJDK: + comment: 'OpenJDK builds (all archs)' + location: 'http://ppa.launchpad.net/openjdk-r/ppa/ubuntu' + release: 'precise' + +cassandra::java::package_name: 'openjdk-8-jdk' diff --git a/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/14.04.yaml b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/14.04.yaml new file mode 100644 index 0000000..bec821f --- /dev/null +++ b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/14.04.yaml @@ -0,0 +1,16 @@ +--- +cassandra::java::aptkey: + OpenJDK: + id: 'DA1A4A13543B466853BAF164EB9B1D8886F44E2A' + server: 'keyserver.ubuntu.com' + +cassandra::java::aptsource: + OpenJDK: + comment: 'OpenJDK builds (all archs)' + location: 'http://ppa.launchpad.net/openjdk-r/ppa/ubuntu' + release: 'trusty' + +cassandra::java::package_name: 'openjdk-8-jdk' + +cassandra::system::swapoff::device: '/dev/mapper/localhost--vg-swap_1' +cassandra::system::swapoff::mount: 'none' diff --git a/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/16.04.yaml b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/16.04.yaml new file mode 100644 index 0000000..62ce000 --- /dev/null +++ b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/16.04.yaml @@ -0,0 +1,15 @@ +--- +cassandra::hints_directory_mode: '0770' + +cassandra::java::aptkey: + OpenJDK: + id: 'DA1A4A13543B466853BAF164EB9B1D8886F44E2A' + server: 'keyserver.ubuntu.com' + +cassandra::java::aptsource: + OpenJDK: + comment: 'OpenJDK builds (all archs)' + location: 'http://ppa.launchpad.net/openjdk-r/ppa/ubuntu' + release: 'xenial' + +cassandra::java::package_name: 'openjdk-8-jdk' diff --git a/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/7.yaml b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/7.yaml new file mode 100644 index 0000000..94149c6 --- /dev/null +++ b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/7.yaml @@ -0,0 +1,17 @@ +--- +cassandra::java::aptkey: + ZuluJDK: + id: '27BC0C8CB3D81623F59BDADCB1998361219BD9C9' + server: 'keyserver.ubuntu.com' + +cassandra::java::aptsource: + ZuluJDK: + location: 'http://repos.azulsystems.com/debian' + comment: 'Zulu OpenJDK 8 for Debian' + release: 'stable' + repos: 'main' + +cassandra::java::package_name: 'zulu-8' + +cassandra::system::swapoff::device: '/dev/mapper/localhost--vg-swap_1' +cassandra::system::swapoff::mount: 'none' diff --git a/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/8.yaml b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/8.yaml new file mode 100644 index 0000000..49706f4 --- /dev/null +++ b/spec/acceptance/hieradata/environments/production/data/osfamily/Debian/8.yaml @@ -0,0 +1,14 @@ +--- +cassandra::java::aptkey: + ZuluJDK: + id: '27BC0C8CB3D81623F59BDADCB1998361219BD9C9' + server: 'keyserver.ubuntu.com' + +cassandra::java::aptsource: + ZuluJDK: + location: 'http://repos.azulsystems.com/debian' + comment: 'Zulu OpenJDK 8 for Debian' + release: 'stable' + repos: 'main' + +cassandra::java::package_name: 'zulu-8' diff --git a/spec/acceptance/hieradata/operatingsystem/CentOS-6.yaml b/spec/acceptance/hieradata/operatingsystem/CentOS-6.yaml index 22b2013..037d2a3 100644 --- a/spec/acceptance/hieradata/operatingsystem/CentOS-6.yaml +++ b/spec/acceptance/hieradata/operatingsystem/CentOS-6.yaml @@ -1,2 +1,7 @@ --- cassandra::run_schema_tests: false +cassandra::system::swapoff::device: '/dev/mapper/VolGroup-lv_swap' +cassandra::system::sysctl::sysctl_file: '/etc/sysctl.conf' +cassandra::system::sysctl::net_ipv4_tcp_rmem: '4096 87380 16777216' +cassandra::system::sysctl::net_ipv4_tcp_wmem: '4096 65536 16777216' +cassandra::system::sysctl::sysctl_args: '-e -p' diff --git a/spec/acceptance/hieradata/operatingsystem/CentOS-7.yaml b/spec/acceptance/hieradata/operatingsystem/CentOS-7.yaml new file mode 100644 index 0000000..6ddcd8d --- /dev/null +++ b/spec/acceptance/hieradata/operatingsystem/CentOS-7.yaml @@ -0,0 +1,5 @@ +--- +cassandra::system::swapoff::device: '/dev/mapper/centos-swap' +cassandra::system::sysctl::net_ipv4_tcp_rmem: '4096, 87380, 16777216' +cassandra::system::sysctl::net_ipv4_tcp_wmem: '4096, 65536, 16777216' +cassandra::system::sysctl::sysctl_file: '/etc/sysctl.d/10-cassandra.conf' diff --git a/spec/acceptance/hieradata/operatingsystem/Ubuntu-12.04.yaml b/spec/acceptance/hieradata/operatingsystem/Ubuntu-12.04.yaml index 7cf65e7..7f408ec 100644 --- a/spec/acceptance/hieradata/operatingsystem/Ubuntu-12.04.yaml +++ b/spec/acceptance/hieradata/operatingsystem/Ubuntu-12.04.yaml @@ -1,6 +1,9 @@ --- cassandra::java::aptsource: OpenJDK: comment: 'OpenJDK builds (all archs)' location: 'http://ppa.launchpad.net/openjdk-r/ppa/ubuntu' release: 'precise' + +cassandra::system::sysctl::net_ipv4_tcp_rmem: '4096 87380 16777216' +cassandra::system::sysctl::net_ipv4_tcp_wmem: '4096 65536 16777216' diff --git a/spec/acceptance/nodesets/centos6.yml b/spec/acceptance/nodesets/centos6.yml index 08bd0a9..adcd3cb 100644 --- a/spec/acceptance/nodesets/centos6.yml +++ b/spec/acceptance/nodesets/centos6.yml @@ -1,17 +1,19 @@ HOSTS: centos6: roles: - centos6 - nodes platform: el-6-x86_64 image: centos:6 hypervisor: docker docker_cmd: '["/sbin/init"]' docker_image_commands: - 'yum -y install gcc tar yum-utils centos-release-scl' - 'yum-config-manager --enable rhel-server-rhscl-7-rpms' - 'yum -y install ruby200 python27' - - '. /opt/rh/python27/enable && /opt/rh/python27/root/usr/bin/pip install cqlsh' + - 'cp /opt/rh/python27/enable /etc/profile.d/python.sh' + - 'echo "" >> /etc/profile.d/python.sh' + - 'echo "export PYTHONPATH=/usr/lib/python2.7/site-packages" >> /etc/profile.d/python.sh' CONFIG: type: foss log_level: notice diff --git a/spec/acceptance/nodesets/ubuntu1404.yml b/spec/acceptance/nodesets/ubuntu1404.yml index 9fc5cca..74bd10c 100644 --- a/spec/acceptance/nodesets/ubuntu1404.yml +++ b/spec/acceptance/nodesets/ubuntu1404.yml @@ -1,14 +1,18 @@ +--- +# Nasty hack with systemd due to issues found in #357. HOSTS: ubuntu1404: roles: - master - nodes - ubuntu14 platform: ubuntu-14.04-x64 image: ubuntu:14.04 hypervisor: docker docker_image_commands: - - 'apt-get install -y iptables sudo' + - 'apt-get install -y iptables sudo systemd' + - 'rm /bin/systemctl' + - 'ln -s /bin/true /bin/systemctl' CONFIG: type: foss log_level: notice diff --git a/spec/acceptance/nodesets/ubuntu1604.yml b/spec/acceptance/nodesets/ubuntu1604.yml index 1a57dff..723022e 100644 --- a/spec/acceptance/nodesets/ubuntu1604.yml +++ b/spec/acceptance/nodesets/ubuntu1604.yml @@ -1,18 +1,21 @@ HOSTS: ubuntu1604: roles: - master - ubuntu16 default_apply_opts: order: random strict_variables: platform: ubuntu-16.04-x64 image: ubuntu:16.04 hypervisor: docker docker_image_commands: - 'apt-get install -y locales-all net-tools sudo ufw wget ntp python-pip python-minimal' - 'rm -f /usr/sbin/policy-rc.d' + - '/usr/bin/wget http://launchpadlibrarian.net/109052632/python-support_1.0.15_all.deb' + - '/usr/bin/dpkg -i python-support_1.0.15_all.deb' + - '/usr/bin/pip install cassandra-driver' docker_cmd: '["/sbin/init"]' CONFIG: type: foss log_level: debug diff --git a/spec/classes/datastax_agent_spec.rb b/spec/classes/datastax_agent_spec.rb index 81e0b78..660181e 100644 --- a/spec/classes/datastax_agent_spec.rb +++ b/spec/classes/datastax_agent_spec.rb @@ -1,137 +1,141 @@ require 'spec_helper' describe 'cassandra::datastax_agent' do let(:pre_condition) do [ 'class cassandra() {}', 'define ini_setting($ensure = nil, $path, $section, $key_val_separator = nil, $setting, $value = nil) {}' ] end context 'Test for cassandra::datastax_agent with defaults (RedHat).' do let :facts do { - osfamily: 'RedHat' + osfamily: 'RedHat', + operatingsystemmajrelease: 6 } end it do should have_resource_count(4) should contain_class('cassandra::datastax_agent').only_with( 'defaults_file' => '/etc/default/datastax-agent', 'java_home' => nil, 'package_ensure' => 'present', 'package_name' => 'datastax-agent', 'service_ensure' => 'running', 'service_enable' => true, 'service_name' => 'datastax-agent', 'stomp_interface' => nil, 'local_interface' => nil ) should contain_package('datastax-agent').with( ensure: 'present', notify: 'Exec[datastax_agent_reload_systemctl]' ).that_notifies('Exec[datastax_agent_reload_systemctl]') should contain_exec('datastax_agent_reload_systemctl').only_with( command: '/usr/bin/systemctl daemon-reload', onlyif: 'test -x /usr/bin/systemctl', path: ['/usr/bin', '/bin'], refreshonly: true, notify: 'Service[datastax-agent]' ).that_notifies('Service[datastax-agent]') should contain_file('/var/lib/datastax-agent/conf/address.yaml') .with( owner: 'cassandra', group: 'cassandra', mode: '0644' ).that_requires('Package[datastax-agent]') should contain_service('datastax-agent').only_with( ensure: 'running', enable: true, name: 'datastax-agent' ) end end context 'Test for cassandra::datastax_agent with defaults (Debian).' do let :facts do { - osfamily: 'Debian' + osfamily: 'Debian', + operatingsystemmajrelease: 6 } end it do should contain_exec('datastax_agent_reload_systemctl').with( command: '/bin/systemctl daemon-reload', onlyif: 'test -x /bin/systemctl', path: ['/usr/bin', '/bin'], refreshonly: true ).that_notifies('Service[datastax-agent]') end end context 'Test that the JAVA_HOME can be set.' do let :facts do { - osfamily: 'Debian' + osfamily: 'Debian', + operatingsystemmajrelease: 6 } end let :params do { java_home: '/usr/lib/jvm/java-8-oracle' } end it do should contain_ini_setting('java_home').with( ensure: 'present', path: '/etc/default/datastax-agent', section: '', key_val_separator: '=', setting: 'JAVA_HOME', value: '/usr/lib/jvm/java-8-oracle' ).that_notifies('Service[datastax-agent]') end end context 'Test settings.' do let :facts do { - osfamily: 'Debian' + osfamily: 'Debian', + operatingsystemmajrelease: 6 } end let :params do { settings: { 'agent_alias' => { 'setting' => 'agent_alias', 'value' => 'foobar' }, 'stomp_interface' => { 'setting' => 'stomp_interface', 'value' => 'localhost' }, 'async_pool_size' => { 'ensure' => 'absent' } } } end it do should have_resource_count(4) end end end diff --git a/spec/classes/init_spec.rb b/spec/classes/init_spec.rb index 01aa88f..9184038 100644 --- a/spec/classes/init_spec.rb +++ b/spec/classes/init_spec.rb @@ -1,404 +1,405 @@ require 'spec_helper' describe 'cassandra' do let(:pre_condition) do [ 'class apt () {}', 'class apt::update () {}', 'define apt::key ($id, $source) {}', 'define apt::source ($location, $comment, $release, $include) {}', 'define ini_setting($ensure = nil, $path, $section, $key_val_separator = nil, $setting, $value = nil) {}' ] end context 'On an unknown OS with defaults for all parameters' do let :facts do { operatingsystemmajrelease: 10, osfamily: 'Darwin' } end it { should raise_error(Puppet::Error) } end context 'Test the default parameters (RedHat)' do let :facts do { osfamily: 'RedHat', operatingsystemmajrelease: 7 } end it do should contain_package('cassandra').with( ensure: 'present', name: 'cassandra22' ).that_notifies('Exec[cassandra_reload_systemctl]') should contain_exec('cassandra_reload_systemctl').only_with( command: '/usr/bin/systemctl daemon-reload', onlyif: 'test -x /usr/bin/systemctl', path: ['/usr/bin', '/bin'], refreshonly: true ) should contain_file('/etc/cassandra/default.conf').with( ensure: 'directory', group: 'cassandra', owner: 'cassandra', mode: '0755' ).that_requires('Package[cassandra]') should contain_file('/etc/cassandra/default.conf/cassandra.yaml') .with( ensure: 'present', owner: 'cassandra', group: 'cassandra', mode: '0644' ) .that_requires('Package[cassandra]') should contain_class('cassandra').only_with( + baseline_settings: {}, cassandra_2356_sleep_seconds: 5, cassandra_9822: false, cassandra_yaml_tmpl: 'cassandra/cassandra.yaml.erb', commitlog_directory_mode: '0750', config_file_mode: '0644', config_path: '/etc/cassandra/default.conf', data_file_directories_mode: '0750', dc: 'DC1', dc_suffix: nil, fail_on_non_supported_os: true, hints_directory_mode: '0750', package_ensure: 'present', package_name: 'cassandra22', rack: 'RAC1', rackdc_tmpl: 'cassandra/cassandra-rackdc.properties.erb', saved_caches_directory_mode: '0750', service_enable: true, # service_ensure: nil, service_name: 'cassandra', service_provider: nil, service_refresh: true, settings: {}, systemctl: '/usr/bin/systemctl' ) end end context 'On RedHat 7 with data directories specified.' do let :facts do { osfamily: 'RedHat', operatingsystemmajrelease: 7 } end let :params do { commitlog_directory: '/var/lib/cassandra/commitlog', data_file_directories: ['/var/lib/cassandra/data'], hints_directory: '/var/lib/cassandra/hints', saved_caches_directory: '/var/lib/cassandra/saved_caches', settings: { 'cluster_name' => 'MyCassandraCluster' } } end it do should have_resource_count(10) should contain_file('/var/lib/cassandra/commitlog') should contain_file('/var/lib/cassandra/data') should contain_file('/var/lib/cassandra/hints') should contain_file('/var/lib/cassandra/saved_caches') end end context 'On RedHat 7 with service provider set to init.' do let :facts do { osfamily: 'RedHat', operatingsystemmajrelease: 7 } end let :params do { service_provider: 'init' } end it do should have_resource_count(7) should contain_exec('/sbin/chkconfig --add cassandra').with( unless: '/sbin/chkconfig --list cassandra' ) .that_requires('Package[cassandra]') .that_comes_before('Service[cassandra]') end end context 'On a Debian OS with defaults for all parameters' do let :facts do { operatingsystemmajrelease: 8, osfamily: 'Debian' } end it do should contain_class('cassandra') should contain_group('cassandra').with_ensure('present') should contain_package('cassandra').with( ensure: 'present', name: 'cassandra' ).that_notifies('Exec[cassandra_reload_systemctl]') should contain_exec('cassandra_reload_systemctl').only_with( command: '/bin/systemctl daemon-reload', onlyif: 'test -x /bin/systemctl', path: ['/usr/bin', '/bin'], refreshonly: true ) should contain_service('cassandra').with( ensure: nil, name: 'cassandra', enable: 'true' ) should contain_exec('CASSANDRA-2356 sleep') .with( command: '/bin/sleep 5', refreshonly: true, user: 'root' ) .that_subscribes_to('Package[cassandra]') .that_comes_before('Service[cassandra]') should contain_user('cassandra') .with( ensure: 'present', comment: 'Cassandra database,,,', gid: 'cassandra', home: '/var/lib/cassandra', shell: '/bin/false', managehome: true ) .that_requires('Group[cassandra]') should contain_file('/etc/cassandra').with( ensure: 'directory', group: 'cassandra', owner: 'cassandra', mode: '0755' ) should contain_file('/etc/cassandra/cassandra.yaml') .with( ensure: 'present', owner: 'cassandra', group: 'cassandra', mode: '0644' ) .that_comes_before('Package[cassandra]') .that_requires(['User[cassandra]', 'File[/etc/cassandra]']) should contain_file('/etc/cassandra/cassandra-rackdc.properties') .with( ensure: 'file', owner: 'cassandra', group: 'cassandra', mode: '0644' ) .that_requires(['File[/etc/cassandra]', 'User[cassandra]']) .that_comes_before('Package[cassandra]') should contain_service('cassandra') .that_subscribes_to( [ 'File[/etc/cassandra/cassandra.yaml]', 'File[/etc/cassandra/cassandra-rackdc.properties]', 'Package[cassandra]' ] ) end end context 'CASSANDRA-9822 activated on Debian' do let :facts do { operatingsystemmajrelease: 7, osfamily: 'Debian', lsbdistid: 'Ubuntu', lsbdistrelease: '14.04' } end let :params do { cassandra_9822: true } end it do should contain_file('/etc/init.d/cassandra').with( source: 'puppet:///modules/cassandra/CASSANDRA-9822/cassandra', mode: '0555' ).that_comes_before('Package[cassandra]') end end context 'Install DSE on a Red Hat family OS.' do let :facts do { operatingsystemmajrelease: 7, osfamily: 'RedHat' } end let :params do { package_ensure: '4.7.0-1', package_name: 'dse-full', config_path: '/etc/dse/cassandra', service_name: 'dse' } end it do should contain_file('/etc/dse/cassandra/cassandra.yaml').that_notifies('Service[cassandra]') should contain_file('/etc/dse/cassandra') should contain_file('/etc/dse/cassandra/cassandra-rackdc.properties') .with( ensure: 'file', owner: 'cassandra', group: 'cassandra', mode: '0644' ) .that_notifies('Service[cassandra]') should contain_package('cassandra').with( ensure: '4.7.0-1', name: 'dse-full' ) is_expected.to contain_service('cassandra').with_name('dse') end end context 'On an unsupported OS pleading tolerance' do let :facts do { operatingsystemmajrelease: 10, osfamily: 'Darwin' } end let :params do { config_file_mode: '0755', config_path: '/etc/cassandra', fail_on_non_supported_os: false, package_name: 'cassandra', service_provider: 'base', systemctl: '/bin/true' } end it do should contain_file('/etc/cassandra/cassandra.yaml').with('mode' => '0755') should contain_service('cassandra').with(provider: 'base') should have_resource_count(6) end end context 'Ensure cassandra service can be stopped and disabled.' do let :facts do { operatingsystemmajrelease: 8, osfamily: 'Debian' } end let :params do { service_ensure: 'stopped', service_enable: 'false' } end it do should contain_service('cassandra') .with(ensure: 'stopped', name: 'cassandra', enable: 'false') end end context 'Test the dc and rack properties with defaults (Debian).' do let :facts do { operatingsystemmajrelease: 8, osfamily: 'Debian' } end it do should contain_file('/etc/cassandra/cassandra-rackdc.properties') .with_content(/^dc=DC1/) .with_content(/^rack=RAC1$/) .with_content(/^#dc_suffix=$/) .with_content(/^# prefer_local=true$/) end end context 'Test the dc and rack properties with defaults (RedHat).' do let :facts do { operatingsystemmajrelease: 7, osfamily: 'RedHat' } end it do should contain_file('/etc/cassandra/default.conf/cassandra-rackdc.properties') .with_content(/^dc=DC1/) .with_content(/^rack=RAC1$/) .with_content(/^#dc_suffix=$/) .with_content(/^# prefer_local=true$/) end end context 'Test the dc and rack properties.' do let :facts do { operatingsystemmajrelease: 7, osfamily: 'RedHat' } end let :params do { snitch_properties_file: 'cassandra-topology.properties', dc: 'NYC', rack: 'R101', dc_suffix: '_1_cassandra', prefer_local: 'true' } end it do should contain_file('/etc/cassandra/default.conf/cassandra-topology.properties') .with_content(/^dc=NYC$/) .with_content(/^rack=R101$/) .with_content(/^dc_suffix=_1_cassandra$/) .with_content(/^prefer_local=true$/) end end end diff --git a/spec/classes/params_spec.rb b/spec/classes/params_spec.rb index b6b87ad..4a152d0 100644 --- a/spec/classes/params_spec.rb +++ b/spec/classes/params_spec.rb @@ -1,15 +1,16 @@ require 'spec_helper' describe '::cassandra::params' do let :facts do { - osfamily: 'RedHat' + osfamily: 'RedHat', + operatingsystemmajrelease: 7 } end it do should compile should contain_class('cassandra::params') should have_resource_count(0) end end diff --git a/spec/classes/system/swapoff_spec.rb b/spec/classes/system/swapoff_spec.rb new file mode 100644 index 0000000..5b95738 --- /dev/null +++ b/spec/classes/system/swapoff_spec.rb @@ -0,0 +1,30 @@ +require 'spec_helper' + +describe 'cassandra::system::swapoff' do + context 'Test the default parameters' do + it do + should have_resource_count(1) + should contain_class('cassandra::system::swapoff') + should contain_exec('Disable Swap') + end + end + + context 'Test we can remove a swap device from /etc/fstab' do + let :params do + { + device: '/dev/mapper/centos-swap' + } + end + + it do + should have_resource_count(2) + should contain_class('cassandra::system::swapoff') + should contain_exec('Disable Swap') + should contain_mount('swap').with( + ensure: 'absent', + device: '/dev/mapper/centos-swap', + fstype: 'swap' + ) + end + end +end diff --git a/spec/classes/system/sysctl_spec.rb b/spec/classes/system/sysctl_spec.rb new file mode 100644 index 0000000..b5199a2 --- /dev/null +++ b/spec/classes/system/sysctl_spec.rb @@ -0,0 +1,39 @@ +require 'spec_helper' + +describe 'cassandra::system::sysctl' do + let(:pre_condition) do + [ + 'define ini_setting($ensure = nil, + $path, + $section, + $key_val_separator = nil, + $setting, + $value = nil) {}' + ] + end + + context 'Test the default parameters (RedHat)' do + let :facts do + { + osfamily: 'RedHat', + operatingsystemmajrelease: 7 + } + end + + it do + should have_resource_count(9) + should contain_class('Cassandra::System::Sysctl') + should contain_ini_setting('net.core.optmem_max = 40960') + should contain_ini_setting('net.core.rmem_default = 16777216') + should contain_ini_setting('net.core.rmem_max = 16777216') + should contain_ini_setting('net.core.wmem_default = 16777216') + should contain_ini_setting('net.core.wmem_max = 16777216') + should contain_ini_setting('net.ipv4.tcp_rmem = 4096, 87380, 16777216') + should contain_ini_setting('net.ipv4.tcp_wmem = 4096, 65536, 16777216') + should contain_ini_setting('vm.max_map_count = 1048575') + should contain_exec('Apply sysctl changes').with( + command: '/sbin/sysctl -p /etc/sysctl.d/10-cassandra.conf' + ) + end + end +end diff --git a/spec/classes/system/transparent_hugepage_spec.rb b/spec/classes/system/transparent_hugepage_spec.rb new file mode 100644 index 0000000..b7df548 --- /dev/null +++ b/spec/classes/system/transparent_hugepage_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' + +describe 'cassandra::system::transparent_hugepage' do + context 'Test the default parameters (RedHat)' do + let :facts do + { + osfamily: 'RedHat', + operatingsystemmajrelease: 7 + } + end + + it do + should have_resource_count(1) + should contain_class('cassandra::system::transparent_hugepage') + should contain_exec('Disable Java Hugepages') + end + end + + context 'Test the default parameters (Debian)' do + let :facts do + { + osfamily: 'Debian', + operatingsystemmajrelease: 7 + } + end + + it do + should have_resource_count(1) + should contain_class('cassandra::system::transparent_hugepage') + should contain_exec('Disable Java Hugepages') + end + end +end diff --git a/spec/defines/file_spec.rb b/spec/defines/file_spec.rb index be54f17..c5d9342 100644 --- a/spec/defines/file_spec.rb +++ b/spec/defines/file_spec.rb @@ -1,51 +1,52 @@ require 'spec_helper' describe '::cassandra::file' do let(:pre_condition) do [ 'class stdlib () {}', 'define ini_setting($ensure = nil, $path, $section, $key_val_separator = nil, $setting, $value = nil) {}', 'define file_line($line, $path, $match) {}' ] end context 'On a Debian OS set the max and new heap size' do let :facts do { - osfamily: 'Debian' + osfamily: 'Debian', + operatingsystemmajrelease: 8 } end let(:title) { 'cassandra-env.sh' } let :params do { config_path: '/etc/cassandra', 'file_lines' => { 'MAX_HEAP_SIZE 4GB' => { 'line' => 'MAX_HEAP_SIZE="4G"', 'match' => '^#MAX_HEAP_SIZE="4G"$' } } } end it do should contain_class('cassandra') should contain_class('cassandra::params') should contain_class('stdlib') should contain_cassandra__file('cassandra-env.sh') should contain_file_line('MAX_HEAP_SIZE 4GB').with( path: '/etc/cassandra/cassandra-env.sh', line: 'MAX_HEAP_SIZE="4G"', match: '^#MAX_HEAP_SIZE="4G"$' ) end end end diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index d80d827..45407aa 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -1,42 +1,57 @@ require 'beaker-rspec' require 'pry' CASSANDRA2_UNSUPPORTED_PLATFORMS = ['16.04'].freeze +thr = Thread.new do + loop do + sleep 1 + `sudo pkill agetty` + end +end + hosts.each do |host| case host.name when 'ubuntu1604' host.install_package('puppet') else install_puppet_on(host) end end RSpec.configure do |c| module_root = File.expand_path(File.join(File.dirname(__FILE__), '..')) c.formatter = :documentation # Configure all nodes in nodeset c.before :suite do # Install modules puppet_module_install(source: module_root, module_name: 'cassandra') hosts.each do |host| on host, puppet('module', 'install', 'puppetlabs-apt'), acceptable_exit_codes: [0, 1] on host, puppet('module', 'install', 'puppetlabs-firewall'), acceptable_exit_codes: [0, 1] on host, puppet('module', 'install', 'puppetlabs-inifile'), acceptable_exit_codes: [0, 1] on host, puppet('module', 'install', 'puppetlabs-stdlib'), acceptable_exit_codes: [0, 1] # Install hiera write_hiera_config_on(host, [ - 'operatingsystem/%{operatingsystem}-%{operatingsystemmajrelease}', - 'operatingsystem/%{operatingsystem}', - 'common' + 'environments/%{environment}/data/fqdn/%{fqdn}', + 'environments/%{environment}/data/osfamily/%{osfamily}/%{lsbdistcodename}', + 'environments/%{environment}/data/osfamily/%{osfamily}/%{lsbmajdistrelease}', + 'environments/%{environment}/data/osfamily/%{osfamily}/%{architecture}', + 'environments/%{environment}/data/osfamily/%{osfamily}/common', + # 'environments/%{environment}/data/modules/%{cname}', + 'environments/%{environment}/data/modules/%{caller_module_name}', + 'environments/%{environment}/data/modules/%{module_name}', + 'environments/%{environment}/data/common' ]) copy_hiera_data_to(host, './spec/acceptance/hieradata/') end end end + +thr.exit diff --git a/templates/cassandra.yaml.erb b/templates/cassandra.yaml.erb index 7d8258d..ad0acfd 100755 --- a/templates/cassandra.yaml.erb +++ b/templates/cassandra.yaml.erb @@ -1,5 +1,5 @@ # This file is managed by Puppet. Manual changes are likely to be # overwritten. If you see that the YAML indentation looks somewhat strange, -# don't worry, please see the following ticker for more details. +# don't worry, please see the following ticket for more details. # https://tickets.puppetlabs.com/browse/PUP-3120 <%= @merged_settings.to_yaml() %> diff --git a/vagrant/environment.conf b/vagrant/environment.conf new file mode 100644 index 0000000..6e6366c --- /dev/null +++ b/vagrant/environment.conf @@ -0,0 +1,2 @@ +# hardcoded module path is "fine" because we're in the "vagrant" directory in our source tree... +modulepath = $codedir/environments/vagrant/modules diff --git a/vagrant/hiera.yaml b/vagrant/hiera.yaml new file mode 100644 index 0000000..fd43ba7 --- /dev/null +++ b/vagrant/hiera.yaml @@ -0,0 +1,9 @@ +--- +:backends: + - yaml +:yaml: + :datadir: /etc/puppetlabs/code/environments/vagrant/hieradata +:hierarchy: + - operatingsystem/%{operatingsystem}-%{operatingsystemmajrelease} + - operatingsystem/%{operatingsystem} + - common diff --git a/vagrant/manifests/site.pp b/vagrant/manifests/site.pp new file mode 100644 index 0000000..af86469 --- /dev/null +++ b/vagrant/manifests/site.pp @@ -0,0 +1,3 @@ +include cassandra::system::transparent_hugepage +include cassandra::system::swapoff +include cassandra::system::sysctl