diff --git a/CHANGELOG.md b/CHANGELOG.md index bf88683..0699fe2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,119 +1,119 @@ -## 0.9.0-UNRELEASED +## 1.0.0 - New features: - [Support Puppet 6](https://github.com/deric/puppet-zookeeper/issues/132) (#132) - [allow configuring additional log appenders (#139)](https://github.com/deric/puppet-zookeeper/pull/139) - Changes: - Drop support for Puppet 3 ([#144](https://github.com/deric/puppet-zookeeper/pull/144)) - Drop support for Puppet 4 ([#145](https://github.com/deric/puppet-zookeeper/pull/145)) - Drop support for Debian (6, 7) and Ubuntu (10.04, 12.04, 14.04) ([#143](https://github.com/deric/puppet-zookeeper/pull/143)) - Fixes: - Fixed and extended acceptance tests - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.8.7...v0.9.0) ## 0.8.7 - New features: - [Support global outstanding limit](https://github.com/deric/puppet-zookeeper/issues/128) (#128) - Fixes: - Fixed and extended acceptance tests - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.8.6...v0.8.7) ## 0.8.6 - New features: - [Support commands whitelist](https://github.com/deric/puppet-zookeeper/issues/135) (#135) - [Support proxy_server for archive install method](https://github.com/deric/puppet-zookeeper/issues/133) (#133) - Fixes: - Fixed mirror URI for ZooKeeper 3.5.5 from archive (d33cf0289333846775860751d12533e72f284c3e) - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.8.5...v0.8.6) ## 0.8.5 - Replace 'require' function with an explicit relationship (#125) - Allow puppetlabs/stdlib 6.x, puppet/archive 4.x (#131) - Use mirror according to required version (#129) - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.8.4...v0.8.5) ## 0.8.4 - Bugfix: prevent service from failing to log any upcoming events ([#122](https://github.com/deric/puppet-zookeeper/pull/122)) - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.8.2...v0.8.4) ## 0.8.3 - Same as `v0.8.2` (incorrectly tagged release) ## 0.8.2 - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.8.1...v0.8.2) - Fixed `JAVA_OPTS` broken with enabled SASL auth ([#120](https://github.com/deric/puppet-zookeeper/pull/120)) ## 0.8.1 * Allow newer dependencies stdlib, archive, puppet (#116) * Include SASL parameters in init.pp (#115) ## 0.8.0 * Dropped Puppet 3 support (still might work with `FUTURE_PARSER=true`) ## 0.7.7 - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.7.6...v0.7.7) - Drop testing on Ruby 1.9.3 - Include Puppet 5 in build matrix - Fixes: - [Replace sh with bash in systemd unit](https://github.com/deric/puppet-zookeeper/pull/113) #113 ## 0.7.6 - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.7.5...v0.7.6) - Fixes: - [zookeeper does not start under debian stretch](https://github.com/deric/puppet-zookeeper/issues/109) #109 ## 0.7.5 - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.7.4...v0.7.5) - New features: - Support Debian 9 - [Allow configuring log file size](https://github.com/deric/puppet-zookeeper/pull/105) - [Allow customizing systed path](https://github.com/deric/puppet-zookeeper/commit/96ae6ee6fd398249d9218c8b242ac39d950bdd9a) - Fixes: - [Properly configure log dir](https://github.com/deric/puppet-zookeeper/issues/108) #108 - [Fix systemd wants directive](https://github.com/deric/puppet-zookeeper/issues/103) #103 - [replace all resource-style class with plain `include`](https://github.com/deric/puppet-zookeeper/pull/106) ## 0.7.4 - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.7.3...v0.7.4) - New features: - [Allow service restarts to be skipped on change](https://github.com/deric/puppet-zookeeper/pull/100) #100 - [Provide option to remove host and realm from Kerberos principal](https://github.com/deric/puppet-zookeeper/pull/99) - Fixes: - [systemd classpath overwritten](https://github.com/deric/puppet-zookeeper/issues/101) #102 ## 0.7.3 - New features: - [Enable possibility of system users for zookeeper](https://github.com/deric/puppet-zookeeper/pull/98) #98 - Fixes: - [Make service run even if bin/zkEnv.sh does not exist](https://github.com/deric/puppet-zookeeper/pull/94) #94 - [Ensure that `/etc/zookeeper` directory exists when installing from archive](https://github.com/deric/puppet-zookeeper/issues/96) #96 - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.7.2...v0.7.3) ## 0.7.2 - New or changed features: - [Basic acceptance tests](https://github.com/deric/puppet-zookeeper/issues/90) - Other changes and fixes: - [Service filename should be equal to the service name](https://github.com/deric/puppet-zookeeper/pull/91) - [Fix service settings when installing from archive](https://github.com/deric/puppet-zookeeper/pull/92) - [Full diff](https://github.com/deric/puppet-zookeeper/compare/v0.7.1...v0.7.2) ## 0.7.1 * New or changed features: * [Exihibitor support](https://github.com/deric/puppet-zookeeper/pull/85) * Other changes and fixes: * [Fixed duplicate java.env declaration when using SASL #89](https://github.com/deric/puppet-zookeeper/issues/89) * [Dependency cycle with archive install method #87](https://github.com/deric/puppet-zookeeper/issues/87) ## 0.7.0 * New or changed features: * Support installation from source package instead of binary package * Ability to install arbitrary versions of Zookeeper from source packages * Other changes and fixes: * Remove deprecated parameters `service_package`, `start_with`, `manage_systemd` * Use correct value for parameter `zoo_dir` in `zookeeper::service` * Move all parameters to `params.pp` (and remove them from private classes) * Overall improvements to the code (style, simplification, deduplication) * Compatibility warnings: * All classes except the main `zookeeper` class are private classes and should not be used directly * Fails when using removed parameters `service_package`, `start_with`, `manage_systemd` diff --git a/README.md b/README.md index 63c1114..ceb2556 100644 --- a/README.md +++ b/README.md @@ -1,380 +1,380 @@ # puppet-zookeeper [![Puppet Forge](http://img.shields.io/puppetforge/v/deric/zookeeper.svg)](https://forge.puppetlabs.com/deric/zookeeper) [![Build Status](https://travis-ci.org/deric/puppet-zookeeper.png?branch=master)](https://travis-ci.org/deric/puppet-zookeeper) [![Puppet Forge Downloads](http://img.shields.io/puppetforge/dt/deric/zookeeper.svg)](https://forge.puppetlabs.com/deric/zookeeper/scores) A puppet receipt for [Apache Zookeeper](http://zookeeper.apache.org/). ZooKeeper is a high-performance coordination service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. ## Requirements * Puppet * Binary or ZooKeeper source code archive Compatibility matrix: | `puppet-zookeeper`| Puppet 3.x | Puppet 4.x | Puppet 5.x | Puppet 6.x | | ----------------- | ------------- |--------------| -----------|----------------| | `0.7.x` | :heavy_check_mark: | :heavy_check_mark: | :question: | :x: | | `0.8.x` | :x: | :heavy_check_mark: | :heavy_check_mark: | :x: | -| `0.9.x` | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | +| `1.0.x` | :x: | :x: | :heavy_check_mark: | :heavy_check_mark: | ## Basic Usage: ```puppet class { 'zookeeper': } ``` ## Cluster setup When running ZooKeeper in the distributed mode each node must have unique ID (`1-255`). The easiest way how to setup multiple ZooKeepers, is by using Hiera. `hiera/host/zk1.example.com.yaml`: ```yaml zookeeper::id: '1' ``` `hiera/host/zk2.example.com.yaml`: ```yaml zookeeper::id: '2' ``` `hiera/host/zk3.example.com.yaml`: ```yaml zookeeper::id: '3' ``` A ZooKeeper quorum should consist of odd number of nodes (usually `3` or `5`). For defining a quorum it is enough to list all IP addresses of all its members. ```puppet class { 'zookeeper': servers => { 1 => '192.168.1.1', 2 => '192.168.1.2', 3 => '192.168.1.3', }, } ``` In case that an array is passed as `servers`, first ZooKeeper will be assigned `ID = 1`. This would produce following configuration: ``` server.1=192.168.1.1:2888:3888 server.2=192.168.1.2:2888:3888 server.3=192.168.1.3:2888:3888 ``` where first port is `election_port` and second one `leader_port`. Both ports could be customized for each ZooKeeper instance. ```puppet class { 'zookeeper': election_port => 2889, leader_port => 3889, servers => { 1 => '192.168.1.1', 2 => '192.168.1.2', 3 => '192.168.1.3', } } ``` ### Observers [Observers](http://zookeeper.apache.org/doc/r3.3.0/zookeeperObservers.html) were introduced in ZooKeeper 3.3.0. To enable this feature simply state which of ZooKeeper servers are observing: ```puppet class { 'zookeeper': servers => ['192.168.1.1', '192.168.1.2', '192.168.1.3', '192.168.1.4', '192.168.1.5'], observers => ['192.168.1.4', '192.168.1.5'] } ``` **Note**: Currently observer server needs to be listed between standard servers (this behavior might change in feature). ### Set binding interface By default ZooKeeper should bind to all interfaces. When you specify `client_ip` only single interface will be used. If `$::ipaddress` is not your public IP (e.g. you are using Docker) make sure to setup correct IP: ```puppet class { 'zookeeper': client_ip => $::ipaddress_eth0 } ``` or in Hiera: ```yaml zookeeper::client_ip: "%{::ipaddress_eth0}" ``` This is a workaround for a a [Facter issue](https://tickets.puppetlabs.com/browse/FACT-380). ### ZooKeeper service Use `service_provider` to override Puppet detection for starting service. ```puppet class { 'zookeeper': service_provider => 'init', manage_service_file => false, } ``` Some reasonable values are: * `init` * `systemd` * `runit` * `exhibitor` - zookeeper process and config will be managed by exhibitor (https://github.com/soabase/exhibitor). Exhibitor is not managed by this module. * `none` - service won't be installed Parameter `manage_service_file` controls whether service definition should be managed by Puppet (default: `false`). Currently supported for `systemd` and `init`. ### Systemd Unit 'After' and 'Want' control By default the module will create the following Unit section in /etc/systemd/system/multi-user.target.wants/zookeeper.service ```` [Unit] Description=Apache ZooKeeper After=network.target ```` Both After and Want (omitted when using the module defaults) can be controled using this module. E.g on CentOS 7 those might have to be configured for 'netwrok-online.target' using the following syntax: ```puppet class { 'zookeeper': systemd_unit_after => 'network-online.target', systemd_unit_want => 'network-online.target', } ``` Which will modify the Unit section to look like: ```` [Unit] Description=Apache ZooKeeper Want=network-online.target After=network-online.target ```` ## Parameters - `id` - cluster-unique zookeeper's instance id (1-255) - `datastore` - `datalogstore` - specifying this configures the `dataLogDir` ZooKeeper config values and allows for transaction logs to be stored in a different location, improving IO performance - `log_dir` - `purge_interval` - automatically will delete ZooKeeper logs (available since ZooKeeper 3.4.0) - `snap_retain_count` - number of snapshots that will be kept after purging (since ZooKeeper 3.4.0) - `min_session_timeout` - the minimum session timeout in milliseconds that the server will allow the client to negotiate. Defaults to 2 times the **tickTime** (since ZooKeeper 3.3.0) - `max_session_timeout` - the maximum session timeout in milliseconds that the server will allow the client to negotiate. Defaults to 20 times the **tickTime** (since ZooKeeper 3.3.0) - `global_outstanding_limit` - ZooKeeper will throttle clients so that there is no more than `global_outstanding_limit` outstanding requests in the system. - `manage_service` (default: `true`) whether Puppet should ensure running service - `manage_service_file` when enabled on RHEL 7.0 a systemd config will be managed - `ensure_account` controls whether `zookeeper` user and group will be ensured (set to `false` to disable this feature) - `install_method` controls whether ZooKeeper is installed from binary (`package`) or source (`archive`) packages - `archive_version` allows to specify an arbitrary version of ZooKeeper when using source packages - `archive_install_dir` controls the installation directory when using source packages (defaults to `/opt`) - `archive_symlink` controls the name of a version-independent symlink when using source packages - `archive_dl_url` allows to change the download URL for source packages (defaults to apache.org) - `systemd_path` where to put `systemd` service files (applies only if `manage_service_file` and `service_provider == 'systemd'`) - `restart_on_change` whether ZooKeeper service should be restarted on configuration files change (default: `true`) - `remove_host_principal` whether to remove host from Kerberos principal (default: `false`) - `remove_realm_principal` whether to remove relam from Kerberos principal (default: `false`) - `whitelist_4lw` Fine grained control over the set of commands ZooKeeper can execute (an array e.g. `whitelist_4lw = ['*']`) and many others, see the `params.pp` file for more details. If your distribution has multiple packages for ZooKeeper, you can provide all package names as an array. ```puppet class { 'zookeeper': packages => ['zookeeper', 'zookeeper-java'] } ``` ## Logging ZooKeeper uses log4j, following variables can be configured: ```puppet class { 'zookeeper': console_threshold => 'INFO', rollingfile_threshold => 'INFO', tracefile_threshold => 'TRACE', maxfilesize => '256MB', maxbackupindex => 20, } ``` Threshold supported values are: `ALL`, `DEBUG`, `ERROR`, `FATAL`, `INFO`, `OFF`, `TRACE` and `WARN`. [Maxfilesize](https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html#maxFileSize) [MaxBackupIndex](https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/RollingFileAppender.html#maxBackupIndex) By default console, rolling file and trace logging can be configured. Additional log appenders (vulgo log methods) can be configured by adding a hash `extra_appenders`. The following sets up syslog logging and points the root logger towards syslog (note that you must have syslog listening on port 514/udp for this to work): ```puppet class { 'zookeeper': log4j_prop => 'INFO,SYSLOG', extra_appenders => { 'Syslog' => { 'class' => 'org.apache.log4j.net.SyslogAppender', 'layout' => 'org.apache.log4j.PatternLayout', 'layout.conversionPattern' => "${hostname} zookeeper[id:%X{myid}] - %-5p [%t:%C{1}@%L][%x] - %m%n", 'syslogHost' => 'localhost', 'facility' => 'user', }, }, } ``` ## Hiera Support All parameters could be defined in hiera files, e.g. `common.yaml`, `Debian.yaml` or `zookeeper.yaml`: ```yaml zookeeper::id: 1 zookeeper::client_port: 2181 zookeeper::datastore: '/var/lib/zookeeper' zookeeper::datalogstore: '/disk2/zookeeper' ``` ## Cloudera package In Cloudera distribution ZooKeeper package does not provide init scripts (same as in Debian). Package containing init scripts is called `zookeeper-server` and the service as well. Moreover there's initialization script which should be called after installation. So, the configuration might look like this: ```puppet class { 'zookeeper': packages => ['zookeeper', 'zookeeper-server'], service_name => 'zookeeper-server', initialize_datastore => true } ``` ### Managing repository For RedHat family currently we support also managing a `cloudera` yum repo versions 4, and 5. It can be enabled with `repo` parameter: ```puppet class { 'zookeeper': repo => 'cloudera', cdhver => '5', } ``` #### Custom RPM repository Optionally you can specify a custom repository, using a hash configuration. ```puppet class { 'zookeeper': cdhver => '5', repo => { name => 'myrepo', url => 'http://cusom.url', descr => 'description' } } ``` ## Source package Source packages provide the ability to install arbitrary versions of ZooKeeper on any platform. Note that you'll likely have to use the `manage_service_file` in order to be able to control the ZooKeeper service (because source packages do not install service files). ```puppet class { 'zookeeper': install_method => 'archive', archive_version => '3.4.8', } ``` Optionally you can specify a `proxy_server`: ```puppet class { 'zookeeper': install_method => 'archive', archive_version => '3.4.8', proxy_server => 'http://10.0.0.1:8080' } ``` ## Java installation Default: `false` By changing these two parameters you can ensure, that given Java package will be installed before ZooKeeper packages. ```puppet class { 'zookeeper': install_java => true, java_package => 'openjdk-7-jre-headless' } ``` ## Install ### Librarian (recommended) For [puppet-librarian](https://github.com/rodjek/librarian-puppet) just add to `Puppetfile` from Forge: ```ruby mod 'deric-zookeeper' ``` latest (development) version from GitHub ```ruby mod 'deric-zookeeper', git: 'git://github.com/deric/puppet-zookeeper.git' ``` ### submodules If you are versioning your puppet conf with git just add it as submodule, from your repository root: git submodule add git://github.com/deric/puppet-zookeeper.git modules/zookeeper ## Dependencies * stdlib `> 2.3.3` - function `ensure_resources` is required * puppet-archive `> 0.4.4` - provides capabilities to use archives instead of binary packages ## Acceptance testing Fastest way is to run tests on prepared Docker images: ``` BEAKER_set=debian9-6.3 bundle exec rake acceptance ``` For examining system state set Beaker's ENV variable `BEAKER_destroy=no`: ``` BEAKER_destroy=no BEAKER_set=default bundle exec rake acceptance ``` and after finishing tests connect to container: ``` docker exec -it adoring_shirley bash ``` When host machine is NOT provisioned (puppet installed, etc.): ``` PUPPET_install=yes BEAKER_set=debian-8 bundle exec rake acceptance ``` Run on specific OS (see `spec/acceptance/nodesets`), to see available sets: ``` rake beaker:sets ``` ## Supported platforms * Debian/Ubuntu * RedHat/CentOS/Fedora ### Tested on: * Debian (8, 9, 10) * Ubuntu (16.04, 18.04) * RHEL (6, 7) * CentOS (6, 7) diff --git a/metadata.json b/metadata.json index b1df1da..1d9c4a4 100644 --- a/metadata.json +++ b/metadata.json @@ -1,66 +1,66 @@ { "name": "deric-zookeeper", - "version": "0.9.0-UNRELEASED", + "version": "1.0.0", "author": "Tomas Barton", "summary": "Module for managing Apache Zookeeper", "license": "Apache-2.0", "source": "https://github.com/deric/puppet-zookeeper", "project_page": "https://github.com/deric/puppet-zookeeper", "issues_url": "https://github.com/deric/puppet-zookeeper/issues", "description": "ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.", "dependencies": [ { "name": "puppet/archive", "version_requirement": ">= 0.4.4 < 5.0.0" }, { "name": "puppet/cron", "version_requirement": ">= 1.3.1" }, { "name": "puppetlabs/stdlib", "version_requirement": ">= 2.3.3 < 7.0.0" }, { "name": "puppetlabs/yumrepo_core", "version_requirement": ">= 1.0.3" } ], "operatingsystem_support": [ { "operatingsystem": "RedHat", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "CentOS", "operatingsystemrelease": [ "6", "7" ] }, { "operatingsystem": "Debian", "operatingsystemrelease": [ "8", "9", "10" ] }, { "operatingsystem": "Ubuntu", "operatingsystemrelease": [ "16.04", "18.04" ] } ], "requirements": [ { "name": "puppet", "version_requirement": ">= 5.5.8 < 7.0.0" } ] }