diff --git a/.gitignore b/.gitignore index 31b263c..35438db 100644 --- a/.gitignore +++ b/.gitignore @@ -1,17 +1,21 @@ .bundle/ coverage/ doc/ Gemfile.lock .idea/ *.iml .librarian/ log/ pkg/ .project Puppetfile.lock spec/fixtures/ *.swn *.swo *.swp +.vagrant/ +vagrant/modules/ vendor/ .yardoc/ +/.vagrant/ +/vagrant/modules/ diff --git a/.rubocop.yml b/.rubocop.yml index d6edbaa..0323ff5 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -1,7 +1,13 @@ --- +AllCops: + Exclude: + - 'vagrant/modules/*/Gemfile' + - 'vagrant/modules/*/Rakefile' + - 'vagrant/modules/**/*' + - 'vendor/**/*' Metrics/BlockLength: Enabled: false Metrics/LineLength: Enabled: false Style/FrozenStringLiteralComment: Enabled: false diff --git a/CHANGELOG.md b/CHANGELOG.md index f43fd4a..adb2164 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,1092 +1,1106 @@ # Change Log for Puppet Module locp-cassandra +##2017-03-30 - Release 2.4.0 ([diff](https://github.com/locp/cassandra/compare/2.3.0...2.4.0)) + +### Summary + +* Added the `cassandra::dse` class to 2.X.Y of the module. +* Added the new `cassandra::apache_repo` class. +* Minor bug fixes and improvements. + +##2017-02-28 - Release 1.27.0 ([diff](https://github.com/locp/cassandra/compare/1.26.1...1.27.0)) + +### Summary + +Add the `cassandra::dse` class. + ##2017-01-21 - Release 2.3.0 ([diff](https://github.com/locp/cassandra/compare/2.2.1...2.3.0)) New attribute to `::cassandra` class: * `baseline_settings` New defined types: * `cassandra::system::swapoff` * `cassandra::system::sysctl` * `cassandra::system::transparent_hugepage` ##2016-12-08 - Release 2.2.1 ([diff](https://github.com/locp/cassandra/compare/2.2.0...2.2.1)) ### Summary Non-functional improvements and some documentation fixes. ### Improvements * All beaker spec tests have been migrated from AWS/EC2 to Docker. * Installed Python 2.7 onto the CentOS 6 docker image allowing the full test suite to be run against that operating system. * Some documentation improvements and fixed some typos. * The spec tests will now fail if there is not 100% coverage. ##2016-12-01 - Release 2.2.0 ([diff](https://github.com/locp/cassandra/compare/2.1.2...2.2.0)) ### Summary At long last we have a defined type for handling permissions (something that we have been promising since May). Added some more facts for assisting in JVM tuning. Also fixed an error that was happening when attempting to build the project. ### Features * Added the `permissions` attribute to the `cassandra::schema` class. * Added the `cassandra::schema::permission` ### Bugfixes * Some refactoring of the ruby code since the release of Rubocop 0.46.0 which was failing code that previously had passed OK. ##2016-11-02 - Release 2.1.2 ([diff](https://github.com/locp/cassandra/compare/2.1.1...2.1.2)) ### Summary Basically a non-functional release to reduce some of the technical debt from the 2.0.0 release. Also some minor improvements to the documentation. ##2016-11-19 - Release 1.26.1 ([diff](https://github.com/locp/cassandra/compare/1.26.0...1.26.1)) ### Summary Retro-fitted a bug that was fixed in 2.1.1 into the 1.X.Y version of the module. ### Bugfixes * A hotfix for a bug in `cassandra::schema` that has been around since version 1.18.0 of this module. ##2016-11-02 - Release 2.1.1 ([diff](https://github.com/locp/cassandra/compare/2.1.0...2.1.1)) ### Summary A hotfix for a bug in `cassandra::schema` that has been around since version 1.18.0 of this module. ### Bugfixes * Fixed a bug where cql_types would not be created unless keyspaces were also provided. * Fixed a minor typo in the README. ##2016-10-29 - Release 2.1.0 ([diff](https://github.com/locp/cassandra/compare/2.0.2...2.1.0)) ### Summary A mixture of new features, a new style of documentation and other improvements. Also ran the [Sonar-Puppet](https://github.com/iwarapter/sonar-puppet) plugin against the code to improve the code quality. ### Features * The following facts are now available: * cassandramajorversion * cassandraminorversion * cassandrapatchversion * cassandrarelease * The following attributes are now available: * cassandra::commitlog_directory * cassandra::commitlog_directory_mode * cassandra::data_file_directories * cassandra::data_file_directories_mode * cassandra::hints_directory * cassandra::hints_directory_mode * cassandra::saved_caches_directory * cassandra::saved_caches_directory_mode * cassandra::systemctl ### Bugfixes * Resolved all issues identified by Sonar-Puppet from 38 (2 major, 36 minor). In doing so, this reduced the technical debt ration from 0.9% to 0.0%. ### Improvements * The reference documentation is now processed via http://locp.github.io/cassandra * Made a start on refactoring the spec/unit tests so that there is less duplication of code. ##2016-10-12 - Release 2.0.2 ([diff](https://github.com/locp/cassandra/compare/2.0.1...2.0.2)) ### Summary This is a non-functional patch release that fixes some issues in documentation, and fixes some bugs and makes some improvements in our testing framework. ### Improvements * Ensure there is a framework for supporting the legacy 1.X.Y. module until the end of 2016. * A more obvious test rig for the beaker/acceptance tests on TravisCI. ### Bugfixes * Corrected misleading documentation. * Ensure legacy test builds still work against Ruby 1.9.3. * Corrected deprecated tags in the module metadata. ##2016-10-08 - Release 1.26.0 ([diff](https://github.com/locp/cassandra/compare/1.25.2...1.26.0)) ### Improvements * The `cassandra::seeds` paramater can now be provided as an array or as a string. # Bugfixes * Ensure legacy builds still work on TravisCI. ##2016-10-01 - Release 2.0.1 ([diff](https://github.com/locp/cassandra/compare/2.0.0...2.0.1)) ### Summary A non-functional release containing some improvements and documentation corrections. ### Improvements * Corrected some out of date documentation. * Removed some deprecated files. * Regained 100% spec test coverage. ##2016-09-29 - Release 2.0.0 ([diff](https://github.com/locp/cassandra/compare/1.26.0...2.0.0)) ### Summary This is a major release and will more than likely break your existing manifest until you alter your code. Please see the [Upgrading](https://github.com/locp/cassandra/tree/release/274/v2.0.0#upgrading) section of the README for further details. * The method of passing parameters to Cassandra has been made more flexible. * Support for Ubuntu 16.04. * Changes to how `service_ensure` is handled. * `cassandra::file` is now a define, not a class (as it should have been). ##2016-09-03 - Release 1.25.2 ([diff](https://github.com/locp/cassandra/compare/1.25.1...1.25.2)) ### Summary The `service_systemd` attribute is now deprecated for the `cassandra`, `cassandra::datastax_agent` and `cassandra::opscenter` classes. See https://github.com/locp/cassandra/wiki/DEP-020 for more details. ##2016-08-14 - Release 1.25.1 ([diff](https://github.com/locp/cassandra/compare/1.25.0...1.25.1)) ### Summary Bugfix to ensure that the name of the PID file for the DataStax Agent service matches with the service name. ##2016-07-17 - Release 1.25.0 ([diff](https://github.com/locp/cassandra/compare/1.24.3...1.25.0)) ### Summary A feature release. ### Features * The `cassandra` class now has a `disk_access_mode` attribute. ##2016-06-26 - Release 1.24.3 ([diff](https://github.com/locp/cassandra/compare/1.24.2...1.24.3)) ### Summary A couple of non-functional bug fixes. ### Bugfixes * The new version of RuboCop flagged code changes required in the Vagrantfile. * The new version of Puppet Lint flagged code changes required in cassandra::java. ##2016-06-20 - Release 1.24.2 ([diff](https://github.com/locp/cassandra/compare/1.24.1...1.24.2)) ### Summary A non-functional bug fix. ### Bugfixes * The release of the specinfra gem (2.59.1) caused problems with our build. Pinned now to 2.59.0. ##2016-06-18 - Release 1.24.1 ([diff](https://github.com/locp/cassandra/compare/1.24.0...1.24.1)) ### Summary Really only affects CentOS 7 as all the other supported operating systems do not require the systemd file to be installed. ### Bugfixes * Change the way that systemd shuts Cassandra down. ##2015-05-25 - Release 1.24.0 ([diff](https://github.com/locp/cassandra/compare/1.23.1...1.24.0)) ### Summary A Debian-centric release. Debian 8 is now supported and a workaround for [CASSANDRA-2356](https://issues.apache.org/jira/browse/CASSANDRA-2356) (which only affects Debian. ### Features * New attributes to the `::cassandra` class are: * cassandra_2356_sleep_seconds * config_path_parents * rackdc_tmpl ### Bugfixes * CASSANDRA-2356 ### Improvements * Debian 8 is now part of the acceptance testing release process. ##2016-05-10 - Release 1.23.1 ([diff](https://github.com/locp/cassandra/compare/1.23.0...1.23.1)) ### Summary A minor bugfix that fixes a problem that would only affect module developers, not users of the module. ### Bugfixes * Rubocop 0.4.0 is stricter so implemented changes that were flagged by it. ##2016-05-07 - Release 1.23.0 ([diff](https://github.com/locp/cassandra/compare/1.22.1...1.23.0)) ### Summary A couple of new features, a non-functional improvement and a bugfix. ### Features * Added the cassandra::file class. * Added the following attributes to the cassandra::schema class: * cqlsh_client_tmpl * cqlsh_client_config ### Bugfixes * Fixed a daft error that stopped beaker tests running on CentOS 7. ### Improvements * Improved how fast the beaker/acceptance tests are run from TravisCI. ##2016-05-03 - Release 1.22.1 ([diff](https://github.com/locp/cassandra/compare/1.22.0...1.22.1)) ### Summary A non-functional release for a minor improvement and a bug fix. ### Bugfixes * Fixes a problem with resource ordering if the service_refresh attribute is set to false. ### Improvements * Test coverage in Coveralls is now back at 100% (https://coveralls.io/github/locp/cassandra). ##2016-04-25 - Release 1.22.0 ([diff](https://github.com/locp/cassandra/compare/1.21.0...1.22.0)) ### Summary Two new features. ### Features * Allow the creation and dropping of users with the following: * A new defined type cassandra::schema::user * The new attribute cassandra::schema::users * Allow the manipulation of the environment file with the new class cassandra::env. ##2016-04-20 - Release 1.21.0 ([diff](https://github.com/locp/cassandra/compare/1.20.0...1.21.0)) ### Summary Some minor bug fixes. Also some features and improvements that allow acceptance tests to be run against Cassandra 3. ### Features * Added the following attributes to cassandra::java: * aptkey * aptsource * yumrepo ### Bugfixes * Corrected some errors in the README and CHANGELOG. * Fixed problems with TravisCI builds. * Fixed problems with CircleCI builds. ### Improvements * Acceptance testing of Cassandra 3. ##2016-04-13 - Release 1.20.0 ([diff](https://github.com/locp/cassandra/compare/1.19.0...1.20.0)) ### Summary A minor release for client requirements. ### Features * Added the compaction_large_partition_warning_threshold_mb and memtable_allocation_type attributes to the cassandra class. ##2016-04-06 - Release 1.19.0 ([diff](https://github.com/locp/cassandra/compare/1.18.1...1.19.0)) ### Summary A new template attribute and a couple of bug fixes. ### Features * The hints_directory attribute has been added to the cassandra class for placing into the template. ### Bugfixes * Some documentation errors that had been identified have been resolved. * A problem with OpsCenter and systemd has been resolved. So far only CentOS 7 from the supported operating systems has been identified as being required to use systemd. ##2016-03-27 - Release 1.18.1 ([diff](https://github.com/locp/cassandra/compare/1.18.0...1.18.1)) ### Summary Bug fixes and some minor and non-functional improvements. ### Bugfixes * Resource ordering clarified in the cassandra::schema class. * The cqlsh command now attempts to connect to rpc_address, not the listen_address. ### Improvements * Optimised the CircleCI build process. * Made some changes to the documentation to better reflect the new functionality delivered in 1.18.0. ##2016-03-26 - Release 1.18.0 ([diff](https://github.com/locp/cassandra/compare/1.17.0...1.18.0)) ### Summary Some more functionality and sub-classes for cassandra::schema. Also some code and pipe-line improvements and a couple of bug fixes. ### Features * The cassandra::schema class now has the following additional attributes: * cql_types * indexes * tables There are also corresponting defined types for those attributes: * cassandra::schema::cql_type * cassandra::schema::index * cassandra::schema::table * The cassandra::opscenter class now has a new attribute called ldap_group_search_filter_with_dn. ### Bugfixes * In the firewalls class, an OpsCenter server also needs to connect to the Cassandra node it is monitoring as a client. The ports have been adjusted to allow this to happen. * rspec-puppet version 2.4.0 was breaking our builds so pegged ourselves to 2.3.2. ### Improvements * Some minor and non-functional improvements to the build pipe-line. * Carried out an audit using the Puppet plugin for SonarQube. The number of issues has been reduced from 227 (13 major, 214 minor) to 4 major issues of which 2 are false positives. The remaining issues will be resolved in 2.0.0. ##2016-03-22 - Release 1.17.0 ([diff](https://github.com/locp/cassandra/compare/1.16.0...1.17.0)) ### Summary Another small change that is an emergency requirement for a client. ### Features * Added the `storage_keyspace` attribute to the cassandra::datastax_agent class. ##2016-03-15 - Release 1.16.0 ([diff](https://github.com/locp/cassandra/compare/1.15.1...1.16.0)) ### Summary A smaller release than usual, but containing changes required for a client. ### Features * Added the `hosts` attribute to the cassandra::datastax_agent class. ### Bugfixes * The **address.yaml** file for the DataStax agent is now owned by the cassandra user. ##2016-03-11 - Release 1.15.1 ([diff](https://github.com/locp/cassandra/compare/1.15.0...1.15.1)) ### Summary A small improvement. ### Improvements * Clarified CQL and Python versions. ##2016-03-10 - Release 1.15.0 ([diff](https://github.com/locp/cassandra/compare/1.14.2...1.15.0)) ### Summary A rather large release. Minor (non-functional) fixes to the production pipeline and new features. ### Features * A new class `cassandra::schema` allows the creation and dropping of keyspaces. * Added the `additional_lines` attribute to the `cassandra` class. * Added the `service_systemd` attribute to the `cassandra::opscenter` class. * Allow the systemd template sources to be specified by the user. This is with the `service_systemd_tmpl` attribute to the following classes: * cassandra * cassandra::datastax_agent * cassandra::opscenter * Added another template file for `cassandra` => `service_systemd_tmpl` that is suitable for Cassandra 2.0. ### Bugfixes * Worked around a problem with rake-11 in the Gemfile. * Fixed a problem in the CircleCI configuration. ### Improvements * Renamed the `ensure` attribute to `package_ensure` so that it is more in line with how it's called in other modules. This was done on the the following classes: * cassandra::optutils * cassandra::opscenter * cassandra::opscenter::pycrypto * cassandra::java * Updated the AWS AMI for the TravisCI/AWS acceptance testing to hopefully speed up builds a bit. ##2016-02-29 - Release 1.14.2 ([diff](https://github.com/locp/cassandra/compare/1.14.1...1.14.2)) ### Summary A small bugfix release. ### Bugfixes * Removed `Restart=always` from the Systemd configuration files. ##2016-02-27 - Release 1.14.1 ([diff](https://github.com/locp/cassandra/compare/1.14.0...1.14.1)) ### Summary A non-functional release of improvements and a bugfix. ### Bugfixes * If a systemd service file is created or updated, then systemctl daemon-reload is now be executed. ### Improvements * The workflow for building the module has been improved to include: * Automatic integration of improvements and bugfixes into release candidates. * Testing of release candidates includes acceptance (beaker) as well as unit (spec) tests. * The refactoring of the majority of the Ruby code used to test and build this module so that it is hopefully more readable and easier to maintain. * The automation of what had previously been manual steps when building a release. ##2016-02-19 - Release 1.14.0 ([diff](https://github.com/locp/cassandra/compare/1.13.0...1.14.0)) ### Summary A minor release with one of each of a feature, bug fix and improvement. ### Features * Added the orbited_longpoll attribute to the cassandra::opscenter class. ### Bugfixes * Fixed a problem with the DataStax agent and systemd. ### Improvements * Refactored the contributors section of the README. ##2016-02-14 - Release 1.13.0 ([diff](https://github.com/locp/cassandra/compare/1.12.2...1.13.0)) ### Summary A mixed back of new features in the shape of attributes for the cassandra, cassandra::datastax_agent and cassandra::opscenter classes. A couple of bug fixes and some non-functional improvements. ### Features * Added the thrift_framed_transport_size_in_mb attribute to ::cassandra. * Added the following attributes to ::cassandra::datastax_agent: * async_pool_size * async_queue_size * service_systemd * Added the config_purge attribute to ::cassandra::opscenter. ### Bugfixes * Removed incorrect puppet code from the README examples. * Fixed a problem with the beaker 2.34.0 gem which was causing problems during automated acceptance testing. ### Improvements * Changed the AWS instance type used by the TravisCI triggered acceptance tests from a c3.xlarge to c4.xlarge. * Merged the acceptance tests virtual nodes into family specific node sets. * Refactored the Gemfile. * Changed references in the documentation to parameters to refer to attributes as that is more Puppet-like. * Changed the format of the contributers section. ##2016-02-12 - Release 1.12.2 ([diff](https://github.com/locp/cassandra/compare/1.12.1...1.12.2)) ### Summary More bug fixes. ### Bugfixes * Fixed a problem with the Red Hat family and systemd not starting the service and reporting all service stops as failures, regardless of if they were or not. ##2016-02-08 - Release 1.12.1 ([diff](https://github.com/locp/cassandra/compare/1.12.0...1.12.1)) ### Summary This is a non-functional release. Some bug fixes and release improvements. ### Bugfixes * Completed documentation for attributes. This was missing for the `inter_dc_stream_throughput_outbound_megabits_per_sec` and `stream_throughput_outbound_megabits_per_sec` options. * Corrected the ownership and directories for the OpsCenter configuration. ### Improvements * Nightly build created so that issues similar to those found in issues #136 and #157 can be caught quicker. ##2016-01-27 - Release 1.12.0 ([diff](https://github.com/locp/cassandra/compare/1.11.0...1.12.0)) ### Summary A new feature in the cassandra::datastax_agent class, a minor bug fix and integration with CircleCI. ### Features * There is now an agent_alias attribute for the cassandra::datastax_agent class. ### Bugfixes * Unit tests were failing due to problems with the puppet-3.8.5 gem. ### Improvements * In addition to TravisCI, the build process is now integrated with [CircleCI](https://circleci.com/gh/locp/cassandra). ##2016-01-01 - Release 1.11.0 ([diff](https://github.com/locp/cassandra/compare/1.10.0...1.11.0)) ### Summary New features added to the main class. Also some non-functional improvements. ### Features * The addition of the listen_interface and rpc_interface attributes to the main class. ### Improvements * Added more detail to the attributes to the main class in the README. * Improved the module metadata. * Clarified private defined types with the private subclass. * The test coverage in release 1.10.0 dropped to 99.09%. Got it back to 100% in this release. ##2015-12-19 - Release 1.10.0 ([diff](https://github.com/locp/cassandra/compare/1.9.2...1.10.0)) ### Summary A feature release with minor improvements. ### Features * Added the ability to allow setting the local_interface for the DataStax agent configuration. * Allow the service provider to be specified for the Cassandra, DataStax agent and OpsCenter services with the service_provider attribute. * Optionally allow a systemd system file be inserted with the cassandra::service_systemd attribute. ### Improvements * Allow the files resources specified with in the cassandra attributes: * commitlog_directory * data_file_directories * saved_caches_directory To co-exist with file resources with the same name. ##2015-11-21 - Release 1.9.2 ([diff](https://github.com/locp/cassandra/compare/1.9.1...1.9.2)) ### Summary A bug fix release that deals with some problems with Cassandra 3. ### Bugfixes * Attempt to mitigate against problems with Debian attempting to install Cassandra 3 when installing the dsc22 package. * Also reverted the project home to the GitHub project page. ##2015-11-09 - Release 1.9.1 ([diff](https://github.com/locp/cassandra/compare/1.9.0...1.9.1)) ### Summary A bug fix release. ### Bugfixes * The default value for the permissions mode of the Cassandra configuration file were far too open. Changed from 0666 to 0644. ##2015-10-25 - Release 1.9.0 ([diff](https://github.com/locp/cassandra/compare/1.8.1...1.9.0)) ### Summary Added more features for the configuration of Cassandra, some improvements to the testing carried out before a release and a minor correction to the change log documentation. ### Features * The following attributes have been added to the ::cassandra class to be configured into the configuration file: * client_encryption_algorithm * client_encryption_cipher_suites * client_encryption_protocol * client_encryption_require_client_auth * client_encryption_store_type * client_encryption_truststore * client_encryption_truststore_password * counter_cache_size_in_mb * index_summary_capacity_in_mb * key_cache_save_period * key_cache_keys_to_save * seed_provider_class_name * server_encryption_algorithm * server_encryption_cipher_suites * server_encryption_protocol * server_encryption_require_client_auth * server_encryption_store_type Please see the README file for more details. ### Bugfixes * Corrected an incorrect date (typo) in this document. ### Improvements * There is now an automated test to mitigate the risk of unnecessarily refreshes of the Cassandra service due to non-functional changes to the configuration file. ##2015-10-14 - Release 1.8.1 ([diff](https://github.com/locp/cassandra/compare/1.8.0...1.8.1)) ### Summary A minor bug fix. ### Bugfixes * Fixed an edge case issue concerning users that may have been using the fail_on_non_supported_os before it was fixed in 1.8.0. ##2015-10-06 - Release 1.8.0 ([diff](https://github.com/locp/cassandra/compare/1.7.1...1.8.0)) ### Summary Some new features a minor bug fix and some non-functional improvements. ### Features * Added the service_refresh and config_file_mode attributes to the Cassandra class. ### Bugfixes * The name of the fail_on_non_supported_os attribute has been corrected. ### Improvements * Automated acceptance tests in preparation for a release now run faster. ##1015-10-01 - Release 1.7.1 ([diff](https://github.com/locp/cassandra/compare/1.7.0...1.7.1)) ### Summary A minor bug fix that incorrctly gave a failed build status for the module. ### Bugfixes * Fixed a problem that was showing the status of the module build as an error since the release of the fog-google gem version 0.1.1. ##2015-10-01 - Release 1.7.0 ([diff](https://github.com/locp/cassandra/compare/1.6.0...1.7.0)) ### Summary * Corrected a bug in how commitlog_sync has handled by Cassandra. * Some non-functional improvements * Additional features for the cassandra::datastax_repo class. ### Features * Added the commitlog_segment_size_in_mb attribute to the cassandra class. * Added the following fields to the cassandra::datastax_repo class: * descr * key_id * key_url * pkg_url * release This should make the configuring of repositories more flexible. ### Bugfixes * Fixed a bug in how the commitlog_sync and the attributes that are associated with it are handled ### Improvements The following non-functional improvements were implemented: * Added tags to the module metadata. * Migrated the acceptance tests from Vagrant to Docker. The associated improvements to performance means that more rigorous acceptance tests can be applied in a shorter time. For the first time as well, they are visible on Travis. ##2015-09-23 - Release 1.6.0 ([diff](https://github.com/locp/cassandra/compare/1.5.0...1.6.0)) ### Summary More attributes for ::cassandra and ::cassandra::datastax_agent. Also some non-functional improvements in the automated unit tests. ### Features * The JAVA_HOME can now be set for the datastax_agent (see the cassandra::datastax_agent => java_home attribute). * The file mode for the directories can now be specified for the commitlog_directory, data_file_directories and the saved_caches_directory in the cassandra class. ### Improvements * Uncovered resources in the unit testing are now tested. ##2015-09-21 - Release 1.5.0 ([diff](https://github.com/locp/cassandra/compare/1.4.2...1.5.0)) ### Summary More attributes have been added that can be configured into the cassandra.yaml file. ### Features * The following attributes to the cassandra class can be configured into the cassandra configuration: * broadcast_address * broadcast_rpc_address * commitlog_sync * commitlog_sync_batch_window_in_ms * commitlog_total_space_in_mb * concurrent_compactors * counter_cache_keys_to_save * file_cache_size_in_mb * initial_token * inter_dc_stream_throughput_outbound_megabits_per_sec * internode_authenticator * internode_recv_buff_size_in_bytes * internode_send_buff_size_in_bytes * memory_allocator * memtable_cleanup_threshold * memtable_flush_writers * memtable_heap_space_in_mb * memtable_offheap_space_in_mb * native_transport_max_concurrent_connections * native_transport_max_concurrent_connections_per_ip * native_transport_max_frame_size_in_mb * native_transport_max_threads * permissions_update_interval_in_ms * phi_convict_threshold * request_scheduler_options_default_weight * request_scheduler_options_throttle_limit * row_cache_keys_to_save * rpc_max_threads * rpc_min_threads * rpc_recv_buff_size_in_bytes * rpc_send_buff_size_in_bytes * streaming_socket_timeout_in_ms * stream_throughput_outbound_megabits_per_sec ### Improvements * Clarity of changes per release in the change log (this document). ##2015-09-15 - Release 1.4.2 ([diff](https://github.com/locp/cassandra/compare/1.4.1...1.4.2)) ### Summary Fixed a problem identified whilst releasing 1.4.1 and a bug fixed by a contributed pull request. ### Features * n/a ### Bugfixes * Fixed a problem with the acceptance tests. * The datastax-agent service is restarted if the package is updated. ### Improvements * n/a ##2015-09-15 - Release 1.4.1 ([diff](https://github.com/locp/cassandra/compare/1.4.0...1.4.1)) ### Summary This release fixes a minor bug (possibly better described as a typing mistake) and makes some non-functional improvements. It also allows the user to override the default behaviour of failing on a non-supported operating system. ### Features * A new flag called `fail_on_non_suppoted_os` has been added to the `cassandra` class and can be set to **false** so that an attempt can be made to use this module on an operating system that is not in the Debian or Red Hat families. ### Bugfixes * Changed the default value for the `package_name` of the `cassandra::optutils` class from `'undef'` to *undef*. ### Improvements * Clarified the expectations of submitted contributions. * Unit test improvements. ##2015-09-10 - Release 1.4.0 ([diff](https://github.com/locp/cassandra/compare/1.3.7...1.4.0)) * Ensured that directories specified in the directory attributes are controlled with file resources. * Added the following attributes to the cassandra.yml file: * batchlog_replay_throttle_in_kb * cas_contention_timeout_in_ms * column_index_size_in_kb * commit_failure_policy * compaction_throughput_mb_per_sec * counter_cache_save_period * counter_write_request_timeout_in_ms * cross_node_timeout * dynamic_snitch_badness_threshold * dynamic_snitch_reset_interval_in_ms * dynamic_snitch_update_interval_in_ms * hinted_handoff_throttle_in_kb * index_summary_resize_interval_in_minutes * inter_dc_tcp_nodelay * max_hints_delivery_threads * max_hint_window_in_ms * permissions_validity_in_ms * range_request_timeout_in_ms * read_request_timeout_in_ms * request_scheduler * request_timeout_in_ms * row_cache_save_period * row_cache_size_in_mb * sstable_preemptive_open_interval_in_mb * tombstone_failure_threshold * tombstone_warn_threshold * trickle_fsync * trickle_fsync_interval_in_kb * truncate_request_timeout_in_ms * write_request_timeout_in_ms ##2015-09-08 - Release 1.3.7 ([diff](https://github.com/locp/cassandra/compare/1.3.6...1.3.7)) * Made the auto_bootstrap attribute available. ##2015-09-03 - Release 1.3.6 ([diff](https://github.com/locp/cassandra/compare/1.3.5...1.3.6)) * Fixed a bug, now allowing the user to set the enabled state of the Cassandra service. * More cleaning up of the README and more links in that file to allow faster navigation. ##2015-09-01 - Release 1.3.5 ([diff](https://github.com/locp/cassandra/compare/1.3.4...1.3.5)) * Fixed a bug, now allowing the user to set the running state of the Cassandra service. * More automated testing with spec tests. * A refactoring of the README. ##2015-08-28 - Release 1.3.4 ([diff](https://github.com/locp/cassandra/compare/1.3.3...1.3.4)) * Minor corrections to the README. * The addition of the storage_cassandra_seed_hosts attribute to cassandra::opscenter::cluster_name which is part of a bigger part of work but is urgently require by a client. ##2015-08-27 - Release 1.3.3 ([diff](https://github.com/locp/cassandra/compare/1.3.2...1.3.3)) * Corrected dependency version for puppetlabs-apt. ##2015-08-26 - Release 1.3.2 ([diff](https://github.com/locp/cassandra/compare/1.3.1...1.3.2)) * Fixed bug in cassandra::opscenter::cluster_name. * Fixed code in cassandra::firewall_ports::rule to avoid deprecation warnings concerning the use of puppetlabs-firewall => port. * Added more examples to the README ##2015-08-22 - Release 1.3.1 ([diff](https://github.com/locp/cassandra/compare/1.3.0...1.3.1)) This was mainly a non-functional change. The biggest thing to say is that Debian 7 is now supported. ##2015-08-19 - Release 1.3.0 ([diff](https://github.com/locp/cassandra/compare/1.2.0...1.3.0)) * Allow additional TCP ports to be specified for the host based firewall. * Fixed a problem where the client subnets were ignored by the firewall. * Added more automated testing. * Continued work on an ongoing improvement of the documentation. * Added the ability to set the DC and RACK in the snitch properties. ##2015-08-10 - Release 1.2.0 ([diff](https://github.com/locp/cassandra/compare/1.1.0...1.2.0)) * Added the installation of Java Native Access (JNA) to cassandra::java * For DataStax Enterprise, allow the remote storage of metric data with cassandra::opscenter::cluster_name. ##2015-08-03 - Release 1.1.0 ([diff](https://github.com/locp/cassandra/compare/1.0.1...1.1.0)) * Provided the cassandra::firewall_ports class. * All OpsCenter options are now configurable in opscenterd.conf. * ssl_storage_port is now configurable. ##2015-07-27 - Release 1.0.1 ([diff](https://github.com/locp/cassandra/compare/1.0.0...1.0.1)) * Provided a workaround for [CASSANDRA-9822](https://issues.apache.org/jira/browse/CASSANDRA-9822). ##2015-07-25 - Release 1.0.0 ([diff](https://github.com/locp/cassandra/compare/0.4.3...1.0.0)) * Changed the default installation from Cassandra 2.1 to 2.2. * Fixed a bug that arose when the cassandra config_path was set. * Created a workaround for [PUP-3829](https://tickets.puppetlabs.com/browse/PUP-3829). * Minor changes to the API (see the Upgrading section of the README). * Allow a basic installation of OpsCenter. ##2015-07-18 - Release 0.4.3 ([diff](https://github.com/locp/cassandra/compare/0.4.2...0.4.3)) * Module dependency metadata was too strict. ##2015-07-16 - Release 0.4.2 ([diff](https://github.com/locp/cassandra/compare/0.4.1...0.4.2)) * Some minor documentation changes. * Fixed a problem with the module metadata that caused Puppetfile issues. * Integrated with Coveralls (https://coveralls.io/github/locp/cassandra). * Removed the deprecated config and install classes. These were private so there is no change to the API. ##2015-07-14 - Release 0.4.1 ([diff](https://github.com/locp/cassandra/compare/0.4.0...0.4.1)) * Fixed a resource ordering problem in the cassandra::datastax class. * Tidied up the documentation a bit. * Some refactoring of the spec tests. ##2015-07-12 - Release 0.4.0 ([diff](https://github.com/locp/cassandra/compare/0.3.0...0.4.0)) ### Summary * Some major changes to the API on how Java, the optional Cassandra tools and the DataStax agent are installed. See the Upgrading section of the README file. * Allowed the setting of the *stomp_interface* for the DataStax agent. * Non-functionally, we have integrated with Travis CI (see https://travis-ci.org/locp/cassandra for details) and thanks to those guys for providing such a neat service. * More spec tests. ##2015-06-27 - Release 0.3.0 ([diff](https://github.com/locp/cassandra/compare/0.2.2...0.3.0)) ### Summary * Slight changes to the API. See the Upgrading section of the README file for full details. * Allow for the installation of the DataStax Agent. * Improved automated testing (and fixed some bugs along the way). * Confirmed Ubuntu 12.04 works OK with this module. * A Cassandra 1.X template has been provided. * Some smarter handling of the differences between Ubuntu/Debian and RedHat derivatives. ##2015-06-17 - Release 0.2.2 ([diff](https://github.com/locp/cassandra/compare/0.2.1...0.2.2)) ### Summary A non-functional change to change the following: * Split the single manifest into multiple files. * Implement automated testing. * Test on additional operating systems. ##2015-05-28 - Release 0.2.1 ([diff](https://github.com/locp/cassandra/compare/0.2.0...0.2.1)) ### Summary A non-functional change to fix puppet-lint problems identified by Puppet Forge. ##2015-05-28 - Release 0.2.0 ([diff](https://github.com/locp/cassandra/compare/0.1.0...0.2.0)) ### Summary Added more attributes and improved the module metadata. ##2015-05-26 - Release 0.1.0 ### Summary An initial release with **VERY** limited options. diff --git a/README.md b/README.md index 80d324b..53defc1 100644 --- a/README.md +++ b/README.md @@ -1,633 +1,669 @@ # Cassandra [![Puppet Forge](http://img.shields.io/puppetforge/v/locp/cassandra.svg)](https://forge.puppetlabs.com/locp/cassandra) [![Github Tag](https://img.shields.io/github/tag/locp/cassandra.svg)](https://github.com/locp/cassandra) [![Build Status](https://travis-ci.org/locp/cassandra.png?branch=master)](https://travis-ci.org/locp/cassandra) [![Coverage Status](https://coveralls.io/repos/locp/cassandra/badge.svg?branch=master&service=github)](https://coveralls.io/github/locp/cassandra?branch=master) [![Join the chat at https://gitter.im/locp/cassandra](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/locp/cassandra?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![CircleCI](https://circleci.com/gh/locp/cassandra.svg?style=svg)](https://circleci.com/gh/locp/cassandra) [![Puppet Forge Downloads](http://img.shields.io/puppetforge/dt/locp/cassandra.svg)](https://forge.puppetlabs.com/locp/cassandra) [![Puppet Forge Endorsement](https://img.shields.io/puppetforge/e/locp/cassandra.svg)](https://forge.puppetlabs.com/locp/cassandra) ## Table of Contents 1. [Overview](#overview) 1. [Setup - The basics of getting started with Cassandra](#setup) * [What Cassandra affects](#what-cassandra-affects) * [Upgrading](#upgrading) * [Beginning with Cassandra](#beginning-with-cassandra) 1. [Usage - Configuration options and additional functionality](#usage) * [Setup a keyspace and users](#setup-a-keyspace-and-users) * [Create a Cluster in a Single Data Center](#create-a-cluster-in-a-single-data-center) * [Create a Cluster in Multiple Data Centers](#create-a-cluster-in-multiple-data-centers) * [DataStax Enterprise](#datastax-enterprise) + * [Apache Cassandra](#apache-cassandra) 1. [Reference](#reference) 1. [Limitations - OS compatibility, etc.](#limitations) 1. [Development](#development) ## Overview A Puppet module to install and manage Cassandra, DataStax Agent & OpsCenter ## Setup ### What Cassandra affects #### What the Cassandra class affects * Installs the Cassandra package (default **cassandra22** on Red Hat and **cassandra** on Debian). * Configures settings in `${config_path}/cassandra.yaml`. * On CentOS 7 if the `init` service provider is used, then cassandra is added as a system service. * Optionally ensures that the Cassandra service is enabled and running. * On Debian systems: * Optionally replace ```/etc/init.d/cassandra``` with a workaround for [CASSANDRA-9822](https://issues.apache.org/jira/browse/CASSANDRA-9822). #### What the cassandra::datastax_agent class affects * Optionally installs the DataStax agent. * Optionally sets JAVA_HOME in **/etc/default/datastax-agent**. #### What the cassandra::datastax_repo class affects * Optionally configures a Yum repository to install the Cassandra packages from (on Red Hat). * Optionally configures an Apt repository to install the Cassandra packages from (on Debian). +#### What the cassandra::dse class affects + +* Optionally configures files in the `/etc/dse` directory if one is using + DataStax Enterprise. + #### What the cassandra::firewall_ports class affects * Optionally configures the firewall for the Cassandra related network ports. #### What the cassandra::java class affects * Optionally installs a JRE/JDK package (e.g. java-1.7.0-openjdk) and the Java Native Access (JNA). #### What the cassandra::optutils class affects * Optionally installs the Cassandra support tools (e.g. cassandra22-tools). ### Upgrading We follow [SemVer Versioning](http://semver.org/) and an update of the major release (i.e. from 1.*Y*.*Z* to 2.*Y*.*Z*) will indicate a significant change to the API which will most probably require a change to your manifest. #### Changes in 2.0.0 This is a major change to the API and you will more than likely need to change your manifest to accomodate these changes. The `service_ensure` attribute of the cassandra class now defaults to *undef*, users who do want to manage service status in Puppet can still set it to true. If leaving the value at the default and setting `service_refresh` and `service_enable` to false will mean that the user and not Puppet running will control the running state of Cassandra. This currently works OK on the Red Hat family, but has issues on Debian due to [CASSANDRA-2356](https://issues.apache.org/jira/browse/CASSANDRA-2356) during an initial install or package upgrade. All the functionality relating to OpsCenter has been divested to the [locp/opscenter](https://forge.puppet.com/locp/opscenter) module on Puppet Forge. It should also be noted that the module no longer creates directories for the `data`, `commitlog`, `saved_caches` and for Cassandra 3 the `hints` directory. These resources will now need to be defined in your manifest/profile. For a list of features that have been deprecated in this release, please see https://github.com/locp/cassandra/wiki/Deprecations For details on migrating from the version 1.X.X attributes to the `settings` hash, see -https://github.com/locp/cassandra/wiki/Version-1.X.Y-Template-Defaults-Shown-As-2.X.Y-Hash +(https://github.com/locp/cassandra/wiki/Suggested-Baseline-Settings) Please also see the notes for 2.0.0 in the [CHANGELOG](https://forge.puppet.com/locp/cassandra/changelog). #### Changes in 1.19.0 The hints_directory documentation will cause a change in the cassandra.yaml file regardless of the value you set it to. If you do not wish this to result in a refesh of the Cassandra service, please set service_refresh to false. #### Changes in 1.9.2 Now that Cassandra 3 is available from the DataStax repositories, there is a problem (especially on Debian) with the operating system package manager attempting to install Cassandra 3. This can be mitigated against using something similar to the code in this modules acceptance test. Please note that the default Cassandra package name has now been changed from 'dsc'. See the documentation for cassandra::package_name below for details. ```puppet if $::osfamily == 'RedHat' { $version = '2.2.4-1' } else { $version = '2.2.4' } class { 'cassandra': package_ensure => $version, } ``` #### Changes in 1.8.0 A somewhat embarrassing correction to the spelling of the cassandra::fail_on_non_suppoted_os to cassandra::fail_on_non_supported_os. #### Issues when Upgrading to 1.4.0 Unfortunately both releases 1.3.7 and 1.4.0 have subsequently been found to call a refresh service even when no changes had been made to the underlying configuration. In release 1.8.0 (somewhat belatedly) the service_refresh flag has been introduced to mitigate against similar problems. #### Issues When Upgrading to 1.3.7 * Please see the notes for 1.4.0. #### Changes in 1.0.0 * cassandra::cassandra_package_ensure has been renamed to cassandra::package_ensure. * cassandra::cassandra_package_name has been renamed to cassandra::package_name. #### Changes in 0.4.0 There is now a cassandra::datastax_agent class, therefore: * cassandra::datastax_agent_package_ensure has now been replaced with cassandra::datastax_agent::package_ensure. * cassandra::datastax_agent_service_enable has now been replaced with cassandra::datastax_agent::service_enable. * cassandra::datastax_agent_service_ensure has now been replaced with cassandra::datastax_agent::service_ensure. * cassandra::datastax_agent_package_name has now been replaced with cassandra::datastax_agent::package_name. * cassandra::datastax_agent_service_name has now been replaced with cassandra::datastax_agent::service_name. Likewise now there is a new class for handling the installation of Java: * cassandra::java_package_ensure has now been replaced with cassandra::java::ensure. * cassandra::java_package_name has now been replaced with cassandra::java::package_name. Also there is now a class for installing the optional utilities: * cassandra::cassandra_opt_package_ensure has now been replaced with cassandra::optutils:ensure. * cassandra::cassandra_opt_package_name has now been replaced with cassandra::optutils:package_name. #### Changes in 0.3.0 * cassandra_opt_package_ensure changed from 'present' to undef. * The manage_service option has been replaced with service_enable and service_ensure. ### Beginning with Cassandra Create a Cassandra 2.X cluster called MyCassandraCluster which uses the GossipingPropertyFileSnitch and password authentication. In this very basic example the node itself becomes a seed for the cluster and the credentials will default to a user called cassandra with a password -called of cassandra.. +called of cassandra. ```puppet # Cassandra pre-requisites include cassandra::datastax_repo include cassandra::java class { 'cassandra': settings => { 'authenticator' => 'PasswordAuthenticator', 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'listen_address' => $::ipaddress, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => $::ipaddress, }, ], }, ], 'start_native_transport' => true, }, require => Class['cassandra::datastax_repo', 'cassandra::java'], } ``` +However, **PLEASE** note that this is the **ABSOLUTE MINIMUM** configuration +to get Cassandra up and running but will probably give you a rather badly +configured node. Please see +[Suggested Baseline Settings](https://github.com/locp/cassandra/wiki/Suggested-Baseline-Settings) +for details on making your configuration a lot more robust. For this code to run with version 3.X of Cassandra, the `hints_directory` will also need to be specified: ```puppet ... class { 'cassandra': settings => { ... 'hints_directory' => '/var/lib/cassandra/hints', ... }, require => Class['cassandra::datastax_repo', 'cassandra::java'], } ``` ### Hiera In your top level node classification (usually `common.yaml`), add the settings hash and all the tweaks you want all the clusters to use: ```YAML cassandra::baseline_settings: authenticator: AllowAllAuthenticator authorizer: AllowAllAuthorizer auto_bootstrap: true auto_snapshot: true ... ``` Then, in the individual node classification add the parts which define the cluster: ```YAML cassandra::settings: cluster_name: developer playground cassandra cluster cassandra::dc: Onsite1 cassandra::rack: RAC1 cassandra::package_ensure: 3.0.5-1 cassandra::package_name: cassandra30 -``` +``` ## Usage ### Setup a keyspace and users We assume that authentication has been enabled for the cassandra cluster and we are connecting with the default user name and password ('cassandra/cassandra'). In this example, we create a keyspace (mykeyspace) with a table called 'users' and an index called 'users_lname_idx'. We also add three users (to Cassandra, not the mykeyspace.users table) called spillman, akers and boone while ensuring that a user called lucan is absent. ```puppet class { 'cassandra': ... } class { 'cassandra::schema': cqlsh_password => 'cassandra', cqlsh_user => 'cassandra', cqlsh_host => $::ipaddress, indexes => { 'users_lname_idx' => { table => 'users', keys => 'lname', keyspace => 'mykeyspace', }, }, keyspaces => { 'mykeyspace' => { durable_writes => false, replication_map => { keyspace_class => 'SimpleStrategy', replication_factor => 1, }, } }, permissions => { 'Grant select permissions to spillman to all keyspaces' => { permission_name => 'SELECT', user_name => 'spillman', }, 'Grant modify to to keyspace mykeyspace to akers' => { keyspace_name => 'mykeyspace', permission_name => 'MODIFY', user_name => 'akers', }, 'Grant alter permissions to mykeyspace to boone' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', user_name => 'boone', }, 'Grant ALL permissions to mykeyspace.users to gbennet' => { keyspace_name => 'mykeyspace', permission_name => 'ALTER', table_name => 'users', user_name => 'gbennet', }, }, tables => { 'users' => { columns => { user_id => 'int', fname => 'text', lname => 'text', 'PRIMARY KEY' => '(user_id)', }, keyspace => 'mykeyspace', }, }, users => { 'spillman' => { password => 'Niner27', }, 'akers' => { password => 'Niner2', superuser => true, }, 'boone' => { password => 'Niner75', }, 'gbennet' => { 'password' => 'foobar', }, 'lucan' => { 'ensure' => absent }, }, } ``` ### Create a Cluster in a Single Data Center In the DataStax documentation _Initializing a multiple node cluster (single data center)_ there is a basic example of a six node cluster with two seeds to be created in a single data center spanning two racks. The nodes in the cluster are: **Node Name** | **IP Address** | ---------------|----------------| node0 (seed 1) | 110.82.155.0 | node1 | 110.82.155.1 | node2 | 110.82.155.2 | node3 (seed 2) | 110.82.156.3 | node4 | 110.82.156.4 | node5 | 110.82.156.5 | Each node is configured to use the GossipingPropertyFileSnitch and 256 virtual nodes (vnodes). The name of the cluster is _MyCassandraCluster_. Also, while building the initial cluster, we are setting the auto_bootstrap to false. In this initial example, we are going to expand the example by: * Ensuring that the software is installed via the DataStax Community repository by including `cassandra::datastax_repo`. This needs to be executed before the Cassandra package is installed. * That a suitable Java Runtime environment (JRE) is installed with Java Native Access (JNA) by including `cassandra::java`. This need to be executed before the Cassandra service is started. ```puppet node /^node\d+$/ { class { 'cassandra::datastax_repo': before => Class['cassandra'] } class { 'cassandra::java': before => Class['cassandra'] } class { 'cassandra': settings => { 'authenticator' => 'AllowAllAuthenticator', 'auto_bootstrap' => false, 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'hints_directory' => '/var/lib/cassandra/hints', 'listen_interface' => 'eth1', 'num_tokens' => 256, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => '110.82.155.0,110.82.156.3', }, ], }, ], 'start_native_transport' => true, }, } } ``` The default value for the num_tokens is already 256, but it is included in the example for clarity. Do not forget to either set auto_bootstrap to true or not set the attribute at all after initializing the cluster. ### Create a Cluster in Multiple Data Centers To continue with the examples provided by DataStax, we look at the example for a cluster across multiple data centers . **Node Name** | **IP Address** | **Data Center** | **Rack** | ---------------|----------------|-----------------|----------| node0 (seed 1) | 10.168.66.41 | DC1 | RAC1 | node1 | 10.176.43.66 | DC1 | RAC1 | node2 | 10.168.247.41 | DC1 | RAC1 | node3 (seed 2) | 10.176.170.59 | DC2 | RAC1 | node4 | 10.169.61.170 | DC2 | RAC1 | node5 | 10.169.30.138 | DC2 | RAC1 | For the sake of simplicity, we will confine this example to the nodes: ```puppet node /^node[012]$/ { class { 'cassandra': dc => 'DC1', settings => { 'authenticator' => 'AllowAllAuthenticator', 'auto_bootstrap' => false, 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'hints_directory' => '/var/lib/cassandra/hints', 'listen_interface' => 'eth1', 'num_tokens' => 256, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => '110.82.155.0,110.82.156.3', }, ], }, ], 'start_native_transport' => true, }, } } node /^node[345]$/ { class { 'cassandra': dc => 'DC2', settings => { 'authenticator' => 'AllowAllAuthenticator', 'auto_bootstrap' => false, 'cluster_name' => 'MyCassandraCluster', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'data_file_directories' => ['/var/lib/cassandra/data'], 'endpoint_snitch' => 'GossipingPropertyFileSnitch', 'hints_directory' => '/var/lib/cassandra/hints', 'listen_interface' => 'eth1', 'num_tokens' => 256, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', 'seed_provider' => [ { 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', 'parameters' => [ { 'seeds' => '110.82.155.0,110.82.156.3', }, ], }, ], 'start_native_transport' => true, }, } } ``` We don't need to specify the rack name (with the rack attribute) as RAC1 is the default value. Again, do not forget to either set auto_bootstrap to true or not set the attribute at all after initializing the cluster. +### DataStax Enterprise + +After configuring the relevant repository, the following snippet works on +CentOS 7 to install DSE Cassandra 4.7.0, set the HADOOP_LOG_DIR, set the +DSE_HOME and configure DataStax Enterprise to use LDAP for authentication: + +```puppet +class { 'cassandra::datastax_repo': + descr => 'DataStax Repo for DataStax Enterprise', + pkg_url => 'https://username:password@rpm.datastax.com/enterprise', + before => Class['cassandra'], +} + +class { 'cassandra': + cluster_name => 'MyCassandraCluster', + config_path => '/etc/dse/cassandra', + package_ensure => '4.7.0-1', + package_name => 'dse-full', + service_name => 'dse', + ... +} + +class { 'cassandra::dse': + file_lines => { + 'Set HADOOP_LOG_DIR directory' => { + ensure => present, + path => '/etc/dse/dse-env.sh', + line => 'export HADOOP_LOG_DIR=/var/log/hadoop', + match => '^# export HADOOP_LOG_DIR=', + }, + 'Set DSE_HOME' => { + ensure => present, + path => '/etc/dse/dse-env.sh', + line => 'export DSE_HOME=/usr/share/dse', + match => '^#export DSE_HOME', + }, + }, + settings => { + ldap_options => { + server_host => localhost, + server_port => 389, + search_dn => 'cn=Admin', + search_password => secret, + use_ssl => false, + use_tls => false, + truststore_type => jks, + user_search_base => 'ou=users,dc=example,dc=com', + user_search_filter => '(uid={0})', + credentials_validity_in_ms => 0, + connection_pool => { + max_active => 8, + max_idle => 8, + } + } + } +} +``` + +### Apache Cassandra +DataStax announced in late October 2016 that it was no longer supporting +the community edition of Cassandra or DSC as it was known (see +*[Take a bow Planet +Cassandra]*(http://www.datastax.com/2016/10/take-a-bow-planet-cassandra) +for details). However, the following snippet of code running on Ubuntu +14.04 worked fine without having to change any of the `::cassandra` class +settings: + +```puppet +require cassandra::java +include cassandra::optutils + +class { 'cassandra::apache_repo': + release => '310x', + before => Class['cassandra', 'cassandra::optutils'], +} + +class { 'cassandra': + ... +} +``` + ## Reference -### Public Classes - -* [cassandra](http://locp.github.io/cassandra/puppet_classes/cassandra.html) -* [cassandra::datastax_agent] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Adatastax_agent.html) -* [cassandra::datastax_repo] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Adatastax_repo.html) -* [cassandra::firewall_ports] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Afirewall_ports.html) -* [cassandra::java] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Ajava.html) -* [cassandra::optutils] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Aoptutils.html) -* [cassandra::schema] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Aschema.html) -* [cassandra::system::swapoff] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Asystem_3A_3Aswapoff.html) -* [cassandra::system::sysctl] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Asystem_3A_3Asysctl.html) -* [cassandra::system::transparent_hugepage] - (http://locp.github.io/cassandra/puppet_classes/cassandra_3A_3Asystem_3A_3Atransparent_hugepage.html) - -### Public Defined Types - -* [cassandra::file] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Afile.html) -* [cassandra::schema::cql_type] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Acql_type.html) -* [cassandra::schema::index] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Aindex.html) -* [cassandra::schema::keyspace] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Akeyspace.html) -* [cassandra::schema::permission] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Apermission.html) -* [cassandra::schema::table] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Atable.html) -* [cassandra::schema::user] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aschema_3A_3Auser.html) - -### Private Defined Types - -* [cassandra::private::firewall_ports::rule] - (http://locp.github.io/cassandra/puppet_defined_types/cassandra_3A_3Aprivate_3A_3Afirewall_ports_3A_3Arule.html) - -### Facts - -* [cassandracmsheapnewsize] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandracmsheapnewsize-instance_method) -* [cassandracmsmaxheapsize] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandracmsmaxheapsize-instance_method) -* [cassandraheapnewsize] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandraheapnewsize-instance_method) -* [cassandramajorversion] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandramajorversion-instance_method) -* [cassandramaxheapsize] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandramaxheapsize-instance_method) -* [cassandraminorversion] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandraminorversion-instance_method) -* [cassandrapatchversion] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandrapatchversion-instance_method) -* [cassandrarelease] - (http://locp.github.io/cassandra/top-level-namespace.html#cassandrarelease-instance_method) +The reference documentation is generated using the +[puppet-strings](https://github.com/puppetlabs/puppet-strings) tool. To see +all of it, please go to +[http://locp.github.io/cassandra](http://locp.github.io/cassandra/_index.html). ## Limitations * When using a Ruby version before 1.9.0, the contents of the Cassandra configuration file may change order of elements due to a problem with to_yaml in earlier versions of Ruby. * When creating key spaces, indexes, cql_types and users the settings will only be used to create a new resource if it does not currently exist. If a change is made to the Puppet manifest but the resource already exits, this change will not be reflected. * At the moment the `cassandra::system::transparent_hugepage` does not persist between reboots. +* Currently Apache Cassandra only provides binary packages for Debian so +the `cassandra::apache_repo` is not of much use to Red Hat family users. ## Development Contributions will be gratefully accepted. Please go to the project page, fork the project, make your changes locally and then raise a pull request. Details on how to do this are available at https://guides.github.com/activities/contributing-to-open-source. Please also see the [CONTRIBUTING.md](https://github.com/locp/cassandra/blob/master/CONTRIBUTING.md) page for project specific requirements. ### Additional Contributers For a list of contributers see [CONTRIBUTING.md](https://github.com/locp/cassandra/blob/master/CONTRIBUTING.md) and https://github.com/locp/cassandra/graphs/contributors diff --git a/Rakefile b/Rakefile index 091f7a6..d543e7c 100644 --- a/Rakefile +++ b/Rakefile @@ -1,14 +1,22 @@ require 'metadata-json-lint/rake_task' require 'puppet_blacksmith/rake_tasks' require 'puppet-strings/tasks' require 'puppetlabs_spec_helper/rake_tasks' require 'rubocop/rake_task' if RUBY_VERSION >= '2.0.0' require 'rubygems' # TravisCI does not require the extra module tasks. require_relative 'rake/rake_tasks' # Use a custom pattern with git tag. %s is replaced with the version number. Blacksmith::RakeTask.new do |t| t.tag_pattern = '%s' end + +exclude_paths = [ + 'vagrant/**/*', + 'vendor/bundle/**/*' +] + +PuppetLint.configuration.ignore_paths = exclude_paths +PuppetSyntax.exclude_paths = exclude_paths diff --git a/Vagrantfile b/Vagrantfile index 3e43714..a95e7ba 100644 --- a/Vagrantfile +++ b/Vagrantfile @@ -1,51 +1,59 @@ # TODO: Find out why cassandra::params not working with vagrant and CentOS 6.2 # TODO: Find out why cassandra::params not working with vagrant and CentOS 7.0 # TODO: Find out why cassandra::params not working with vagrant and Ubuntu 12.04. -# TODO: Have vagrant dynamically discover Puppet modules from metadata.json. Vagrant.configure('2') do |config| # config.vm.box = 'puppetlabs/centos-6.6-64-puppet' # config.vm.box = 'puppetlabs/centos-7.0-64-puppet' - config.vm.box = 'puppetlabs/debian-7.8-64-puppet' + # config.vm.box = 'puppetlabs/debian-7.8-64-puppet' # config.vm.box = 'puppetlabs/debian-8.2-64-puppet' # config.vm.box = "puppetlabs/ubuntu-12.04-64-puppet" - # config.vm.box = "puppetlabs/ubuntu-14.04-64-puppet" + config.vm.box = 'puppetlabs/ubuntu-14.04-64-puppet' # config.vm.box = "puppetlabs/ubuntu-16.04-64-puppet" config.vm.provider 'virtualbox' do |vm| vm.memory = 2048 vm.cpus = 2 end puppet_environment = 'vagrant' puppet_environment_path_on_guest = "/etc/puppetlabs/code/environments/#{puppet_environment}" module_path_on_guest = "#{puppet_environment_path_on_guest}/modules" config.vm.synced_folder './vagrant', '/etc/puppetlabs/code/environments/vagrant' config.vm.synced_folder '.', '/etc/puppetlabs/code/environments/vagrant/modules/cassandra' config.vm.synced_folder './spec/acceptance/hieradata', '/etc/puppetlabs/code/environments/vagrant/hieradata' - config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/ || mkdir #{puppet_environment_path_on_guest}" - config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/apt || puppet module install puppetlabs-apt --environment=#{puppet_environment}" - config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/firewall || puppet module install puppetlabs-firewall --environment=#{puppet_environment}" - config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/inifile || puppet module install puppetlabs-inifile --environment=#{puppet_environment}" - config.vm.provision :shell, inline: "test -d #{module_path_on_guest}/stdlib || puppet module install puppetlabs-stdlib --environment=#{puppet_environment}" + metadata_json_file = "#{File.dirname(__FILE__)}/metadata.json" + config.vm.provision :shell, + inline: "test -d #{module_path_on_guest}/ || mkdir #{puppet_environment_path_on_guest}" + + if File.exist?(metadata_json_file) + JSON.parse(File.read(metadata_json_file))['dependencies'].each do |key, _value| + module_name = key['name'].to_s + short_name = module_name.split('-')[1] + config.vm.provision :shell, + inline: "test -d #{module_path_on_guest}/#{short_name} || puppet module install #{module_name} --environment=#{puppet_environment}" + end + else + puts 'metadata.json not found; skipping install of dependencies' + end config.vm.provision :puppet do |puppet| puppet.options = ENV['PUPPET_OPTS'].split(' ') if ENV.key?('PUPPET_OPTS') # See http://stackoverflow.com/a/27540417/224334 puppet.options = '--verbose --debug' if ENV['PUPPET_VERBOSE'] puppet.hiera_config_path = 'vagrant/hiera.yaml' puppet.environment = puppet_environment puppet.environment_path = './' puppet.manifests_path = "#{puppet_environment}/manifests" puppet.manifest_file = 'site.pp' puppet.facter = { project_name: 'ENGLISH NAME Of PROJECT', # EDIT THIS LINE } end config.vm.network :forwarded_port, guest: 22, host: 2223, auto_correct: true, id: 'ssh' - config.vm.network :forwarded_port, guest: 3000, host: 3000 + # config.vm.network :forwarded_port, guest: 3000, host: 3000 end diff --git a/examples/apache.pp b/examples/apache.pp new file mode 100644 index 0000000..180fd02 --- /dev/null +++ b/examples/apache.pp @@ -0,0 +1,4 @@ +# Cassandra pre-requisites +require cassandra::system::sysctl +require cassandra::system::transparent_hugepage +require cassandra::java diff --git a/examples/default_template_settings.pp b/examples/default_template_settings.pp index 7def185..932916f 100644 --- a/examples/default_template_settings.pp +++ b/examples/default_template_settings.pp @@ -1,101 +1,97 @@ # The defaults as set by the 1.Y.Z template. # Cassandra pre-requisites include cassandra::datastax_repo include cassandra::java class { 'cassandra': settings => { 'authenticator' => 'AllowAllAuthenticator', 'authorizer' => 'AllowAllAuthorizer', 'auto_snapshot' => true, 'batch_size_warn_threshold_in_kb' => 5, 'batchlog_replay_throttle_in_kb' => 1024, 'cas_contention_timeout_in_ms' => 1000, 'client_encryption_options' => { 'enabled' => false, 'keystore' => 'conf/.keystore', 'keystore_password' => 'cassandra', }, 'cluster_name' => 'Test Cluster', 'column_index_size_in_kb' => 64, 'commit_failure_policy' => 'stop', 'commitlog_directory' => '/var/lib/cassandra/commitlog', 'commitlog_segment_size_in_mb' => 32, 'commitlog_sync' => 'periodic', 'commitlog_sync_period_in_ms' => 10000, 'compaction_throughput_mb_per_sec' => 16, 'concurrent_counter_writes' => 32, 'concurrent_reads' => 32, 'concurrent_writes' => 32, 'counter_cache_save_period' => 7200, 'counter_cache_size_in_mb' => '', 'counter_write_request_timeout_in_ms' => 5000, 'cross_node_timeout' => false, 'data_file_directories' => ['/var/lib/cassandra/data'], 'disk_failure_policy' => 'stop', 'dynamic_snitch_badness_threshold' => 0.1, 'dynamic_snitch_reset_interval_in_ms' => 600000, 'dynamic_snitch_update_interval_in_ms' => 100, 'endpoint_snitch' => 'SimpleSnitch', 'hinted_handoff_enabled' => true, 'hinted_handoff_throttle_in_kb' => 1024, 'incremental_backups' => false, 'index_summary_capacity_in_mb' => '', 'index_summary_resize_interval_in_minutes' => 60, 'inter_dc_tcp_nodelay' => false, 'internode_compression' => 'all', 'key_cache_save_period' => 14400, 'key_cache_size_in_mb' => '', 'listen_address' => 'localhost', 'max_hint_window_in_ms' => 10800000, 'max_hints_delivery_threads' => 2, 'memtable_allocation_type' => 'heap_buffers', 'native_transport_port' => 9042, 'num_tokens' => 256, 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', 'permissions_validity_in_ms' => 2000, 'range_request_timeout_in_ms' => 10000, 'read_request_timeout_in_ms' => 5000, 'request_scheduler' => 'org.apache.cassandra.scheduler.NoScheduler', 'request_timeout_in_ms' => 10000, 'row_cache_save_period' => 0, 'row_cache_size_in_mb' => 0, 'rpc_address' => 'localhost', 'rpc_keepalive' => true, 'rpc_port' => 9160, 'rpc_server_type' => 'sync', 'saved_caches_directory' => '/var/lib/cassandra/saved_caches', - 'seed_provider' => [ - { - 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', - 'parameters' => [ - { - 'seeds' => '127.0.0.1', - } - ], - } - ], + 'seed_provider' => [{ + 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', + 'parameters' => [{ + 'seeds' => '127.0.0.1', + }], + }], 'server_encryption_options' => { 'internode_encryption' => 'none', 'keystore' => 'conf/.keystore', 'keystore_password' => 'cassandra', 'truststore' => 'conf/.truststore', 'truststore_password' => 'cassandra', }, 'snapshot_before_compaction' => false, 'ssl_storage_port' => 7001, 'sstable_preemptive_open_interval_in_mb' => 50, 'start_native_transport' => true, 'start_rpc' => true, 'storage_port' => 7000, 'thrift_framed_transport_size_in_mb' => 15, 'tombstone_failure_threshold' => 100000, 'tombstone_warn_threshold' => 1000, 'trickle_fsync' => false, 'trickle_fsync_interval_in_kb' => 10240, 'truncate_request_timeout_in_ms' => 60000, 'write_request_timeout_in_ms' => 2000, }, require => Class['cassandra::datastax_repo', 'cassandra::java'], } diff --git a/manifests/apache_repo.pp b/manifests/apache_repo.pp new file mode 100644 index 0000000..f79678f --- /dev/null +++ b/manifests/apache_repo.pp @@ -0,0 +1,69 @@ +# An optional class that will allow a suitable repository to be configured +# from which packages for Apache Cassandra can be downloaded. Changing +# the defaults will allow any Debian Apt repository to be configured. +# Currently Apache only provide a Debian repository +# (http://cassandra.apache.org/download/) mentions of Yum are for wishful +# thinking. +# @param descr [string] On the Red Hat family, this is passed as the `descr` +# attribute to a `yumrepo` resource. On the Debian family, it is passed as +# the `comment` attribute to an `apt::source` resource. +# @param key_id [string] On the Debian family, this is passed as the `id` +# attribute to an `apt::key` resource. On the Red Hat family, it is +# ignored. +# @param key_url [string] On the Debian family, this is passed as the +# `source` attribute to an `apt::key` resource. On the Red Hat family, +# it is ignored. +# @param pkg_url [string] If left as the default, this will set the `baseurl` +# to 'http://rpm.datastax.com/community' on a `yumrepo` resource +# on the Red Hat family. On the Debian family, leaving this as the default +# will set the `location` attribute on an `apt::source` to +# 'http://www.apache.org/dist/cassandra/debian'. +# @param release [string] On the Debian family, this is passed as the `release` +# attribute to an `apt::source` resource. On the Red Hat family, it is +# ignored. +class cassandra::apache_repo ( + $descr = 'Repo for Apache Cassandra', + $key_id = 'A26E528B271F19B9E5D8E19EA278B781FE4B2BDA', + $key_url = 'https://www.apache.org/dist/cassandra/KEYS', + $pkg_url = undef, + $release = 'main', + ) { + case $::osfamily { + 'Debian': { + include apt + include apt::update + + apt::key {'apache.cassandra': + id => $key_id, + source => $key_url, + before => Apt::Source['cassandra.sources'], + } + + if $pkg_url != undef { + $location = $pkg_url + } else { + $location = 'http://www.apache.org/dist/cassandra/debian' + } + + apt::source {'cassandra.sources': + location => $location, + comment => $descr, + release => $release, + include => { + 'src' => false, + }, + notify => Exec['update-apache-cassandra-repo'], + } + + # Required to wrap apt_update + exec {'update-apache-cassandra-repo': + refreshonly => true, + command => '/bin/true', + require => Exec['apt_update'], + } + } + default: { + warning("OS family ${::osfamily} not supported") + } + } +} \ No newline at end of file diff --git a/manifests/dse.pp b/manifests/dse.pp new file mode 100644 index 0000000..54de2e8 --- /dev/null +++ b/manifests/dse.pp @@ -0,0 +1,85 @@ +# A class for configuring DataStax Enterprise (DSE) specific settings. +# +# @param config_file [string] The full path to the DSE configuration file. +# @param config_file_mode [string] The mode for the DSE configuration file. +# @param dse_yaml_tmpl [string] A path to a template for the `dse.yaml` file. +# @param file_lines [hash] A hash of values that are passed to +# `create_resources` as a `file_line` resource. +# @param service_refresh [boolean] Whether or not the Cassandra service +# should be refreshed if the DSE configuration files are changed. +# @param settings [hash] Unless this attribute is set to a hash (which is +# then placed as YAML inside `dse.yaml`) then the `dse.yaml` is left +# unchanged. +# @example Configure a cluster with LDAP authentication +# class { 'cassandra::dse': +# file_lines => { +# 'Set HADOOP_LOG_DIR directory' => { +# ensure => present, +# path => '/etc/dse/dse-env.sh', +# line => 'export HADOOP_LOG_DIR=/var/log/hadoop', +# match => '^# export HADOOP_LOG_DIR=', +# }, +# 'Set DSE_HOME' => { +# ensure => present, +# path => '/etc/dse/dse-env.sh', +# line => 'export DSE_HOME=/usr/share/dse', +# match => '^#export DSE_HOME', +# }, +# }, +# settings => { +# ldap_options => { +# server_host => localhost, +# server_port => 389, +# search_dn => 'cn=Admin', +# search_password => secret, +# use_ssl => false, +# use_tls => false, +# truststore_type => jks, +# user_search_base => 'ou=users,dc=example,dc=com', +# user_search_filter => '(uid={0})', +# credentials_validity_in_ms => 0, +# connection_pool => { +# max_active => 8, +# max_idle => 8, +# } +# } +# } +# } +class cassandra::dse ( + $config_file = '/etc/dse/dse.yaml', + $config_file_mode = '0644', + $dse_yaml_tmpl = 'cassandra/dse.yaml.erb', + $file_lines = undef, + $service_refresh = true, + $settings = undef, + ) { + include cassandra + include stdlib + + if $service_refresh { + $notifications = Service['cassandra'] + } else { + $notifications = [] + } + + if is_hash($file_lines) { + $default_file_line = { + require => Package['cassandra'], + notify => $notifications, + } + + create_resources(file_line, $file_lines, $default_file_line) + } + + if is_hash($settings) { + file { $config_file: + ensure => present, + owner => 'cassandra', + group => 'cassandra', + content => template($dse_yaml_tmpl), + mode => $config_file_mode, + require => Package['cassandra'], + notify => $notifications, + } + } +} diff --git a/metadata.json b/metadata.json index 07f5d6a..2173075 100644 --- a/metadata.json +++ b/metadata.json @@ -1,91 +1,82 @@ { "name": "locp-cassandra", - "version": "2.3.0", + "version": "2.4.0", "author": "locp", - "summary": "Installs Cassandra, DataStax Agent & OpsCenter on RHEL/Ubuntu/Debian.", + "summary": "Installs Cassandra & DataStax Agent on RHEL/Ubuntu/Debian.", "license": "Apache-2.0", "source": "https://github.com/locp/cassandra", "project_page": "https://github.com/locp/cassandra", "issues_url": "https://github.com/locp/cassandra/issues", "tags": [ + "Apache", "cassandra", "cluster", "database", "datastax", "datastax-agent", "nosql" ], - "requirements": [ - { - "name": "pe", - "version_requirement": ">= 3.7.0 < 2015.4.0" - }, - { - "name": "puppet", - "version_requirement": ">= 3.0.0 < 5.0.0" - } - ], "dependencies": [ { "name": "puppetlabs-apt", "version_requirement": ">= 2.0.0 < 3.0.0" }, { "name": "puppetlabs-firewall", "version_requirement": ">= 1.0.0 < 2.0.0" }, { "name": "puppetlabs-inifile", "version_requirement": ">= 1.5.0 < 2.0.0" }, { "name": "puppetlabs-stdlib", "version_requirement": ">= 3.0.0 < 5.0.0" } ], "operatingsystem_support": [ { "operatingsystem": "CentOS", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "OracleLinux", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "RedHat", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "Debian", "operatingsystemrelease": [ "7", "8" ] }, { "operatingsystem": "Scientific", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "Ubuntu", "operatingsystemrelease": [ "12.04", "14.04", "16.04" ] } ] } diff --git a/rake/rake_tasks.rb b/rake/rake_tasks.rb index 85a0300..61a77d2 100644 --- a/rake/rake_tasks.rb +++ b/rake/rake_tasks.rb @@ -1,48 +1,48 @@ ############################################################################# # Some module specific rake tasks. ############################################################################# require 'fileutils' require_relative 'tasks/deploy' desc '[CI Only] Run beaker, but only for pull requests or for release branches.' task :acceptance do travis_pull_request = ENV['TRAVIS_PULL_REQUEST'] if travis_pull_request.nil? || (travis_pull_request == 'false') puts 'Skipping acceptance tests.' exit(0) else Rake::Task['beaker'].invoke end end desc '[CI Only] Tag, build and push the module to PuppetForge.' task :deploy do abort('Only deploy from master.') unless ENV['CIRCLE_BRANCH'] == 'master' # Find out what the local version of the module is. file = File.read('metadata.json') data_hash = JSON.parse(file) local_version = data_hash['version'] abort('Unable to find local module version.') unless local_version puts "Module version (local): #{local_version}" Rake::Task['deploy:tag'].invoke(local_version) Rake::Task['deploy:forge'].invoke(local_version) end desc 'Run metadata_lint, rubocop, lint, validate and spec.' -task test: [ - :metadata_lint, - :rubocop, - :lint, - :validate, - :spec -] +task test: %i( + metadata_lint + rubocop + lint + validate + spec +) desc 'Clean up after a vagrant run.' task :vagrant_clean do module_root = File.expand_path(File.join(__FILE__, '..', '..')) directory = File.expand_path(File.join(module_root, 'vagrant', 'modules')) FileUtils.rm_r directory if File.directory?(directory) end diff --git a/spec/classes/dse_spec.rb b/spec/classes/dse_spec.rb new file mode 100644 index 0000000..ce93cab --- /dev/null +++ b/spec/classes/dse_spec.rb @@ -0,0 +1,28 @@ +require 'spec_helper' +describe 'cassandra::dse' do + let(:pre_condition) do + [ + 'class stdlib () {}', + 'define file_line($line, $path, $match) {}' + ] + end + + context 'with defaults for all parameters' do + let :facts do + { + osfamily: 'RedHat', + operatingsystemmajrelease: 7 + } + end + + it do + should have_resource_count(6) + should contain_class('cassandra') + + should contain_class('cassandra::dse').with( + config_file_mode: '0644', + config_file: '/etc/dse/dse.yaml' + ) + end + end +end diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index 5642999..674fe08 100644 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -1,156 +1,160 @@ require 'rubygems' require 'rspec-puppet' require 'rspec-puppet-utils' require 'puppetlabs_spec_helper/module_spec_helper' require 'simplecov' require 'coveralls' unless ENV['TRAVIS'] == 'true' Coveralls.wear! unless ENV['TRAVIS'] == 'true' fixture_path = File.expand_path(File.join(__FILE__, '..', 'fixtures')) RSpec.configure do |config| config.module_path = File.join(fixture_path, 'modules') config.manifest_dir = File.join(fixture_path, 'manifests') config.tty = true config.mock_with :rspec config.raise_errors_for_deprecations! config.before(:each) do MockFunction.new('concat') do |f| f.stubbed.returns([8888, 22]) f.stubbed.with([], '/etc/cassandra') .returns(['/etc/cassandra']) f.stubbed.with([], '/etc/cassandra/default.conf') .returns(['/etc/cassandra/default.conf']) f.stubbed.with(['/etc/cassandra'], '/etc/cassandra/default.conf') .returns(['/etc/cassandra', '/etc/cassandra/default.conf']) end MockFunction.new('count') do |f| f.stubbed.with( [ 'COMPACT STORAGE', 'ID=\'5a1c395e-b41f-11e5-9f22-ba0be0483c18\'' ] ).returns(2) end MockFunction.new('create_ini_settings', type: :statement) do |f| end MockFunction.new('delete') do |f| f.stubbed.with( { 'keyspace_class' => 'NetworkTopologyStrategy', 'dc1' => '3', 'dc2' => '2' }, 'keyspace_class' ).returns('dc1' => '3', 'dc2' => '2') f.stubbed.with('userid text, username FROZEN, emails set, top_scores list, todo map, COLLECTION-TYPE tuple, PRIMARY KEY (userid)', 'COLLECTION-TYPE ').returns('userid text, username FROZEN, emails set, top_scores list, todo map, tuple, PRIMARY KEY (userid)') end MockFunction.new('is_array') do |f| f.stubbed.with('').returns(false) f.stubbed.with(['/var/lib/cassandra/data']).returns(true) end + MockFunction.new('is_hash') do |f| + f.stubbed.with('').returns(false) + end + MockFunction.new('join') do |f| f.stubbed.with(['firstname text', 'lastname text'], ', ') .returns('firstname text, lastname text') f.stubbed.with( { '\'dc1\': ' => '3', '\'dc2\': ' => '2' }, ', ' ).returns('\'dc1\': 3, \'dc2\': 2') f.stubbed.with( [ 'COMPACT STORAGE', 'ID=\'5a1c395e-b41f-11e5-9f22-ba0be0483c18\'' ], ' AND ' ).returns("COMPACT STORAGE AND ID='5a1c395e-b41f-11e5-9f22-ba0be0483c18'") f.stubbed.with( [ 'userid text', 'username FROZEN', 'emails set', 'top_scores list', 'todo map', 'COLLECTION-TYPE tuple', 'PRIMARY KEY (userid)' ], ', ' ).returns('userid text, username FROZEN, emails set, top_scores list, todo map, COLLECTION-TYPE tuple, PRIMARY KEY (userid)') end MockFunction.new('join_keys_to_values') do |f| f.stubbed.with({ 'firstname' => 'text', 'lastname' => 'text' }, ' ') .returns(['firstname text', 'lastname text']) f.stubbed.with( { '\'dc1' => '3', '\'dc2' => '2' }, '\': ' ).returns('\'dc1\': ' => '3', '\'dc2\': ' => '2') f.stubbed.with( { 'userid' => 'text', 'username' => 'FROZEN', 'emails' => 'set', 'top_scores' => 'list', 'todo' => 'map', 'COLLECTION-TYPE' => 'tuple', 'PRIMARY KEY' => '(userid)' }, ' ' ).returns( [ 'userid text', 'username FROZEN', 'emails set', 'top_scores list', 'todo map', 'COLLECTION-TYPE tuple', 'PRIMARY KEY (userid)' ] ) end MockFunction.new('merge') do |f| end MockFunction.new('prefix') do |f| f.stubbed.with(['0.0.0.0/0'], '200_Public_').returns('200_Public_0.0.0.0/0') f.stubbed.with(['0.0.0.0/0'], '210_InterNode_').returns('210_InterNode__0.0.0.0/0') f.stubbed.with(['0.0.0.0/0'], '220_Client_').returns('220_Client__0.0.0.0/0') f.stubbed.with( { 'dc1' => '3', 'dc2' => '2' }, '\'' ).returns('\'dc1' => '3', '\'dc2' => '2') end MockFunction.new('upcase') do |f| f.stubbed.with('ALL').returns('ALL') f.stubbed.with('ALTER').returns('ALTER') f.stubbed.with('AUTHORIZE').returns('AUTHORIZE') f.stubbed.with('CREATE').returns('CREATE') f.stubbed.with('DROP').returns('DROP') f.stubbed.with('MODIFY').returns('MODIFY') f.stubbed.with('SELECT').returns('SELECT') f.stubbed.with('field').returns('FIELD') f.stubbed.with('forty9ers').returns('FORTY9ERS') f.stubbed.with('ravens').returns('ravens') end MockFunction.new('size') do |f| f.stubbed.returns(42) end MockFunction.new('strftime') do |f| f.stubbed.with('/var/lib/cassandra-%F') .returns('/var/lib/cassandra-YYYY-MM-DD') end MockFunction.new('validate_hash', type: :statement) do |f| end end config.after(:suite) do exit(1) if RSpec::Puppet::Coverage.report!(100) end end diff --git a/templates/dse.yaml.erb b/templates/dse.yaml.erb new file mode 100755 index 0000000..b4666da --- /dev/null +++ b/templates/dse.yaml.erb @@ -0,0 +1,5 @@ +# This file is managed by Puppet. Manual changes are likely to be +# overwritten. If you see that the YAML indentation looks somewhat strange, +# don't worry, please see the following ticket for more details. +# https://tickets.puppetlabs.com/browse/PUP-3120 +<%= @settings.to_yaml() %> diff --git a/vagrant/manifests/site.pp b/vagrant/manifests/site.pp index af86469..50f14c4 100644 --- a/vagrant/manifests/site.pp +++ b/vagrant/manifests/site.pp @@ -1,3 +1,36 @@ -include cassandra::system::transparent_hugepage -include cassandra::system::swapoff -include cassandra::system::sysctl +require cassandra::java +include cassandra::optutils + +class { 'cassandra::apache_repo': + release => '310x', + before => Class['cassandra', 'cassandra::optutils'], +} + +class { 'cassandra': + commitlog_directory => '/var/lib/cassandra/commitlog', + data_file_directories => ['/var/lib/cassandra/data'], + hints_directory => '/var/lib/cassandra/hints', + saved_caches_directory => '/var/lib/cassandra/saved_caches', + settings => { + 'authenticator' => 'PasswordAuthenticator', + 'authorizer' => 'CassandraAuthorizer', + 'cluster_name' => 'MyCassandraCluster', + 'commitlog_sync' => 'periodic', + 'commitlog_sync_period_in_ms' => 10000, + 'endpoint_snitch' => 'GossipingPropertyFileSnitch', + 'listen_address' => $::ipaddress, + 'partitioner' => 'org.apache.cassandra.dht.Murmur3Partitioner', + 'seed_provider' => [ + { + 'class_name' => 'org.apache.cassandra.locator.SimpleSeedProvider', + 'parameters' => [ + { + 'seeds' => $::ipaddress, + }, + ], + }, + ], + 'start_native_transport' => true, + }, + service_ensure => running, +}