diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..ecb10a8 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,15 @@ +# editorconfig.org + +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +root = true + +[*] +charset = utf-8 +end_of_line = lf +indent_size = 2 +tab_width = 2 +indent_style = space +insert_final_newline = true +trim_trailing_whitespace = true diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 0000000..048d2b5 --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,252 @@ +# Contribution guidelines + +## Table of contents + +* [Contributing](#contributing) +* [Writing proper commits - short version](#writing-proper-commits-short-version) +* [Writing proper commits - long version](#writing-proper-commits-long-version) +* [Dependencies](#dependencies) + * [Note for OS X users](#note-for-os-x-users) +* [The test matrix](#the-test-matrix) +* [Syntax and style](#syntax-and-style) +* [Running the unit tests](#running-the-unit-tests) +* [Unit tests in docker](#unit-tests-in-docker) +* [Integration tests](#integration-tests) + +This module has grown over time based on a range of contributions from +people using it. If you follow these contributing guidelines your patch +will likely make it into a release a little more quickly. + +## Contributing + +Please note that this project is released with a Contributor Code of Conduct. +By participating in this project you agree to abide by its terms. +[Contributor Code of Conduct](https://voxpupuli.org/coc/). + +* Fork the repo. +* Create a separate branch for your change. +* We only take pull requests with passing tests, and documentation. [GitHub Actions](https://docs.github.com/en/actions) run the tests for us. You can also execute them locally. This is explained [in a later section](#the-test-matrix). +* Checkout [our docs](https://voxpupuli.org/docs/reviewing_pr/) we use to review a module and the [official styleguide](https://puppet.com/docs/puppet/6.0/style_guide.html). They provide some guidance for new code that might help you before you submit a pull request. +* Add a test for your change. Only refactoring and documentation changes require no new tests. If you are adding functionality or fixing a bug, please add a test. +* Squash your commits down into logical components. Make sure to rebase against our current master. +* Push the branch to your fork and submit a pull request. + +Please be prepared to repeat some of these steps as our contributors review your code. + +Also consider sending in your profile code that calls this component module as an acceptance test or provide it via an issue. This helps reviewers a lot to test your use case and prevents future regressions! + +## Writing proper commits - short version + +* Make commits of logical units. +* Check for unnecessary whitespace with "git diff --check" before committing. +* Commit using Unix line endings (check the settings around "crlf" in git-config(1)). +* Do not check in commented out code or unneeded files. +* The first line of the commit message should be a short description (50 characters is the soft limit, excluding ticket number(s)), and should skip the full stop. +* Associate the issue in the message. The first line should include the issue number in the form "(#XXXX) Rest of message". +* The body should provide a meaningful commit message, which: + *uses the imperative, present tense: `change`, not `changed` or `changes`. + * includes motivation for the change, and contrasts its implementation with the previous behavior. + * Make sure that you have tests for the bug you are fixing, or feature you are adding. + * Make sure the test suites passes after your commit: + * When introducing a new feature, make sure it is properly documented in the README.md + +## Writing proper commits - long version + + 1. Make separate commits for logically separate changes. + + Please break your commits down into logically consistent units + which include new or changed tests relevant to the rest of the + change. The goal of doing this is to make the diff easier to + read for whoever is reviewing your code. In general, the easier + your diff is to read, the more likely someone will be happy to + review it and get it into the code base. + + If you are going to refactor a piece of code, please do so as a + separate commit from your feature or bug fix changes. + + We also really appreciate changes that include tests to make + sure the bug is not re-introduced, and that the feature is not + accidentally broken. + + Describe the technical detail of the change(s). If your + description starts to get too long, that is a good sign that you + probably need to split up your commit into more finely grained + pieces. + + Commits which plainly describe the things which help + reviewers check the patch and future developers understand the + code are much more likely to be merged in with a minimum of + bike-shedding or requested changes. Ideally, the commit message + would include information, and be in a form suitable for + inclusion in the release notes for the version of Puppet that + includes them. + + Please also check that you are not introducing any trailing + whitespace or other "whitespace errors". You can do this by + running "git diff --check" on your changes before you commit. + + 2. Sending your patches + + To submit your changes via a GitHub pull request, we _highly_ + recommend that you have them on a topic branch, instead of + directly on `master`. + It makes things much easier to keep track of, especially if + you decide to work on another thing before your first change + is merged in. + + GitHub has some pretty good + [general documentation](http://help.github.com/) on using + their site. They also have documentation on + [creating pull requests](http://help.github.com/send-pull-requests/). + + In general, after pushing your topic branch up to your + repository on GitHub, you can switch to the branch in the + GitHub UI and click "Pull Request" towards the top of the page + in order to open a pull request. + + + 3. Update the related GitHub issue. + + If there is a GitHub issue associated with the change you + submitted, then you should update the ticket to include the + location of your branch, along with any other commentary you + may wish to make. + +## Dependencies + +The testing and development tools have a bunch of dependencies, +all managed by [bundler](http://bundler.io/) according to the +[Puppet support matrix](http://docs.puppetlabs.com/guides/platforms.html#ruby-versions). + +By default the tests use a baseline version of Puppet. + +If you have Ruby 2.x or want a specific version of Puppet, +you must set an environment variable such as: + +```sh +export PUPPET_VERSION="~> 5.5.6" +``` + +You can install all needed gems for spec tests into the modules directory by +running: + +```sh +bundle install --path .vendor/ --without development system_tests release --jobs "$(nproc)" +``` + +If you also want to run acceptance tests: + +```sh +bundle install --path .vendor/ --with system_tests --without development release --jobs "$(nproc)" +``` + +Our all in one solution if you don't know if you need to install or update gems: + +```sh +bundle install --path .vendor/ --with system_tests --without development release --jobs "$(nproc)"; bundle update; bundle clean +``` + +As an alternative to the `--jobs "$(nproc)` parameter, you can set an +environment variable: + +```sh +BUNDLE_JOBS="$(nproc)" +``` + +### Note for OS X users + +`nproc` isn't a valid command under OS x. As an alternative, you can do: + +```sh +--jobs "$(sysctl -n hw.ncpu)" +``` + +## The test matrix + +### Syntax and style + +The test suite will run [Puppet Lint](http://puppet-lint.com/) and +[Puppet Syntax](https://github.com/gds-operations/puppet-syntax) to +check various syntax and style things. You can run these locally with: + +```sh +bundle exec rake lint +bundle exec rake validate +``` + +It will also run some [Rubocop](http://batsov.com/rubocop/) tests +against it. You can run those locally ahead of time with: + +```sh +bundle exec rake rubocop +``` + +### Running the unit tests + +The unit test suite covers most of the code, as mentioned above please +add tests if you're adding new functionality. If you've not used +[rspec-puppet](http://rspec-puppet.com/) before then feel free to ask +about how best to test your new feature. + +To run the linter, the syntax checker and the unit tests: + +```sh +bundle exec rake test +``` + +To run your all the unit tests + +```sh +bundle exec rake spec +``` + +To run a specific spec test set the `SPEC` variable: + +```sh +bundle exec rake spec SPEC=spec/foo_spec.rb +``` + +#### Unit tests in docker + +Some people don't want to run the dependencies locally or don't want to install +ruby. We ship a Dockerfile that enables you to run all unit tests and linting. +You only need to run: + +```sh +docker build . +``` + +Please ensure that a docker daemon is running and that your user has the +permission to talk to it. You can specify a remote docker host by setting the +`DOCKER_HOST` environment variable. it will copy the content of the module into +the docker image. So it will not work if a Gemfile.lock exists. + +### Integration tests + +The unit tests just check the code runs, not that it does exactly what +we want on a real machine. For that we're using +[beaker](https://github.com/puppetlabs/beaker). + +This fires up a new virtual machine (using vagrant) and runs a series of +simple tests against it after applying the module. You can run this +with: + +```sh +BEAKER_setfile=debian10-x64 bundle exec rake beaker +``` + +You can replace the string `debian10` with any common operating system. +The following strings are known to work: + +* ubuntu1604 +* ubuntu1804 +* ubuntu2004 +* debian9 +* debian10 +* centos7 +* centos8 + +For more information and tips & tricks, see [voxpupuli-acceptance's documentation](https://github.com/voxpupuli/voxpupuli-acceptance#running-tests). + +The source of this file is in our [modulesync_config](https://github.com/voxpupuli/modulesync_config/blob/master/moduleroot/.github/CONTRIBUTING.md.erb) +repository. diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index 1279361..593e7aa 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -1,30 +1,26 @@ +Thank you for contributing to this project! - -* Module version: -* Puppet version: -* OS and version: +## Affected Puppet, Ruby, OS and module versions/distributions -## Bug description +- Puppet: +- Ruby: +- Distribution: +- Module version: - +## How to reproduce (e.g Puppet code you use) + +## What are you seeing + +## What behaviour did you expect instead + +## Output log -## Feature Description +## Any additional information you'd like to impart diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 2b0d43c..342807b 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,9 +1,20 @@ - + +#### Pull Request (PR) description + + +#### This Pull Request (PR) fixes the following issues + diff --git a/.github/SECURITY.md b/.github/SECURITY.md new file mode 100644 index 0000000..cacadf2 --- /dev/null +++ b/.github/SECURITY.md @@ -0,0 +1,3 @@ +# Vox Pupuli Security Policy + +Our vulnerabilities reporting process is at https://voxpupuli.org/security/ diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..d08d05e --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,90 @@ +--- +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +name: CI + +on: pull_request + +concurrency: + group: ${{ github.head_ref }} + cancel-in-progress: true + +jobs: + setup_matrix: + name: 'Setup Test Matrix' + runs-on: ubuntu-latest + timeout-minutes: 40 + outputs: + puppet_unit_test_matrix: ${{ steps.get-outputs.outputs.puppet_unit_test_matrix }} + github_action_test_matrix: ${{ steps.get-outputs.outputs.github_action_test_matrix }} + env: + BUNDLE_WITHOUT: development:system_tests:release + steps: + - uses: actions/checkout@v2 + - name: Setup ruby + uses: ruby/setup-ruby@v1 + with: + ruby-version: '3.0' + bundler-cache: true + - name: Run static validations + run: bundle exec rake validate lint check + - name: Run rake rubocop + run: bundle exec rake rubocop + - name: Setup Test Matrix + id: get-outputs + run: bundle exec metadata2gha --use-fqdn --pidfile-workaround false + + unit: + needs: setup_matrix + runs-on: ubuntu-latest + timeout-minutes: 40 + strategy: + fail-fast: false + matrix: + include: ${{fromJson(needs.setup_matrix.outputs.puppet_unit_test_matrix)}} + env: + BUNDLE_WITHOUT: development:system_tests:release + PUPPET_VERSION: "~> ${{ matrix.puppet }}.0" + name: Puppet ${{ matrix.puppet }} (Ruby ${{ matrix.ruby }}) + steps: + - uses: actions/checkout@v2 + - name: Setup ruby + uses: ruby/setup-ruby@v1 + with: + ruby-version: ${{ matrix.ruby }} + bundler-cache: true + - name: Run tests + run: bundle exec rake parallel_spec + + acceptance: + needs: setup_matrix + runs-on: ubuntu-latest + env: + BUNDLE_WITHOUT: development:test:release + strategy: + fail-fast: false + matrix: + include: ${{fromJson(needs.setup_matrix.outputs.github_action_test_matrix)}} + name: ${{ matrix.puppet.name }} - ${{ matrix.setfile.name }} + steps: + - uses: actions/checkout@v2 + - name: Setup ruby + uses: ruby/setup-ruby@v1 + with: + ruby-version: '3.0' + bundler-cache: true + - name: Run tests + run: bundle exec rake beaker + env: + BEAKER_PUPPET_COLLECTION: ${{ matrix.puppet.collection }} + BEAKER_setfile: ${{ matrix.setfile.value }} + + tests: + needs: + - unit + - acceptance + runs-on: ubuntu-latest + name: Test suite + steps: + - run: echo Test suite completed diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..664ba69 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,34 @@ +--- +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +name: Release + +on: + push: + tags: + - '*' + +env: + BUNDLE_WITHOUT: development:test:system_tests + +jobs: + deploy: + name: 'deploy to forge' + runs-on: ubuntu-latest + if: github.repository_owner == 'voxpupuli' + steps: + - name: Checkout repository + uses: actions/checkout@v2 + - name: Setup Ruby + uses: ruby/setup-ruby@v1 + with: + ruby-version: '2.7' + bundler-cache: true + - name: Build and Deploy + env: + # Configure secrets here: + # https://docs.github.com/en/free-pro-team@latest/actions/reference/encrypted-secrets + BLACKSMITH_FORGE_USERNAME: '${{ secrets.PUPPET_FORGE_USERNAME }}' + BLACKSMITH_FORGE_API_KEY: '${{ secrets.PUPPET_FORGE_API_KEY }}' + run: bundle exec rake module:push diff --git a/.gitignore b/.gitignore index 218e73e..9b95224 100644 --- a/.gitignore +++ b/.gitignore @@ -1,20 +1,23 @@ -*.deb -*.lock -*.rpm -*.sha1 -*.tar.gz -*.zip -.bundle -.ruby-version -.swp -.vagrant -.vendor -.yardoc -doc -log +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + pkg/ -spec/fixtures/manifests -spec/fixtures/modules -spec/logs -spec/reports/ -vendor +Gemfile.lock +Gemfile.local +vendor/ +.vendor/ +spec/fixtures/manifests/ +spec/fixtures/modules/ +.vagrant/ +.bundle/ +.ruby-version +coverage/ +log/ +.idea/ +.dependencies/ +.librarian/ +Puppetfile.lock +*.iml +.*.sw? +.yardoc/ +Guardfile diff --git a/.msync.yml b/.msync.yml new file mode 100644 index 0000000..a83abd9 --- /dev/null +++ b/.msync.yml @@ -0,0 +1,5 @@ +--- +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +modulesync_config_version: '5.1.0' diff --git a/.overcommit.yml b/.overcommit.yml new file mode 100644 index 0000000..d367ada --- /dev/null +++ b/.overcommit.yml @@ -0,0 +1,65 @@ +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ +# +# Hooks are only enabled if you take action. +# +# To enable the hooks run: +# +# ``` +# bundle exec overcommit --install +# # ensure .overcommit.yml does not harm to you and then +# bundle exec overcommit --sign +# ``` +# +# (it will manage the .git/hooks directory): +# +# Examples howto skip a test for a commit or push: +# +# ``` +# SKIP=RuboCop git commit +# SKIP=PuppetLint git commit +# SKIP=RakeTask git push +# ``` +# +# Don't invoke overcommit at all: +# +# ``` +# OVERCOMMIT_DISABLE=1 git commit +# ``` +# +# Read more about overcommit: https://github.com/brigade/overcommit +# +# To manage this config yourself in your module add +# +# ``` +# .overcommit.yml: +# unmanaged: true +# ``` +# +# to your modules .sync.yml config +--- +PreCommit: + RuboCop: + enabled: true + description: 'Runs rubocop on modified files only' + command: ['bundle', 'exec', 'rubocop'] + PuppetLint: + enabled: true + description: 'Runs puppet-lint on modified files only' + command: ['bundle', 'exec', 'puppet-lint'] + YamlSyntax: + enabled: true + JsonSyntax: + enabled: true + TrailingWhitespace: + enabled: true + +PrePush: + RakeTarget: + enabled: true + description: 'Run rake targets' + targets: + - 'validate' + - 'test' + - 'rubocop' + command: ['bundle', 'exec', 'rake'] diff --git a/.pmtignore b/.pmtignore index 2ba009d..65f5051 100644 --- a/.pmtignore +++ b/.pmtignore @@ -1,11 +1,37 @@ -*.swp -*.org +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +docs/ +pkg/ Gemfile Gemfile.lock -Makefile +Gemfile.local +vendor/ +.vendor/ +spec/ Rakefile -doc/ -junit/ +.vagrant/ +.bundle/ +.ruby-version +coverage/ log/ -logs/ -spec/ +.idea/ +.dependencies/ +.github/ +.librarian/ +Puppetfile.lock +*.iml +.editorconfig +.fixtures.yml +.gitignore +.msync.yml +.overcommit.yml +.pmtignore +.rspec +.rspec_parallel +.rubocop.yml +.sync.yml +.*.sw? +.yardoc/ +.yardopts +Dockerfile diff --git a/.puppet-lint.rc b/.puppet-lint.rc new file mode 100644 index 0000000..dd8272c --- /dev/null +++ b/.puppet-lint.rc @@ -0,0 +1,3 @@ +--fail-on-warnings +--no-parameter_documentation-check +--no-parameter_types-check diff --git a/.rspec b/.rspec new file mode 100644 index 0000000..f634583 --- /dev/null +++ b/.rspec @@ -0,0 +1,5 @@ +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +--format documentation +--color diff --git a/.rspec_parallel b/.rspec_parallel new file mode 100644 index 0000000..a9a84f8 --- /dev/null +++ b/.rspec_parallel @@ -0,0 +1,4 @@ +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +--format progress diff --git a/.rubocop.yml b/.rubocop.yml index 4e8824c..53ac189 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -1,66 +1,6 @@ -AllCops: - Exclude: - - pkg/**/* - - spec/fixtures/**/* - - vendor/**/* - Include: - - Rakefile - TargetRubyVersion: 1.9 +--- +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ -Metrics/AbcSize: - Enabled: false - -Metrics/LineLength: - Enabled: false - -Metrics/MethodLength: - Enabled: false - -Metrics/ParameterLists: - Enabled: false - -Style/AndOr: - Enabled: false - -Style/BlockDelimiters: - Enabled: false - -Style/BlockEndNewline: - Enabled: false - -Style/ClassAndModuleCamelCase: - Enabled: false - -Style/ClassAndModuleChildren: - Enabled: false - -Style/FirstParameterIndentation: - Enabled: false - -Style/FrozenStringLiteralComment: - Enabled: false - -Style/HashSyntax: - Enabled: false - -Style/IndentArray: - Enabled: false - -Style/Lambda: - Enabled: false - -Style/MultilineBlockChain: - Enabled: false - -Style/MultilineBlockLayout: - Enabled: false - -Style/Not: - Enabled: false - -# This is the default post-0.41.2 rubocop, set it to be forward-compatible here. -Style/PercentLiteralDelimiters: - PreferredDelimiters: - '%r': '{}' - '%w': '[]' - '%W': '[]' +inherit_gem: + voxpupuli-test: rubocop.yml diff --git a/.sync.yml b/.sync.yml new file mode 100644 index 0000000..ab7e62b --- /dev/null +++ b/.sync.yml @@ -0,0 +1,14 @@ +--- +Gemfile: + optional: + ':test': + - gem: bcrypt + - gem: webmock + ':system_tests': + - gem: bcrypt + - gem: rspec-retry + - gem: infrataster +spec/spec_helper.rb: + hiera_config: "'spec/fixtures/hiera/hiera.yaml'" +spec/spec_helper_acceptance.rb: + unmanaged: false diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..e3cf307 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,24 @@ +# MANAGED BY MODULESYNC +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +FROM ruby:2.7 + +WORKDIR /opt/puppet + +# https://github.com/puppetlabs/puppet/blob/06ad255754a38f22fb3a22c7c4f1e2ce453d01cb/lib/puppet/provider/service/runit.rb#L39 +RUN mkdir -p /etc/sv + +ARG PUPPET_VERSION="~> 6.0" +ARG PARALLEL_TEST_PROCESSORS=4 + +# Cache gems +COPY Gemfile . +RUN bundle install --without system_tests development release --path=${BUNDLE_PATH:-vendor/bundle} + +COPY . . + +RUN bundle install +RUN bundle exec rake release_checks + +# Container should not saved +RUN exit 1 diff --git a/Gemfile b/Gemfile index 6abd16b..fdfb1dd 100644 --- a/Gemfile +++ b/Gemfile @@ -1,58 +1,39 @@ -source ENV['GEM_SOURCE'] || 'https://rubygems.org' +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +source ENV['GEM_SOURCE'] || "https://rubygems.org" group :test do - gem 'puppet', (ENV['PUPPET_VERSION'] || '~> 6.0'), :require => false - - gem 'metadata-json-lint' - gem 'specinfra', '~> 2.60' - gem 'xmlrpc' - - gem 'ci_reporter_rspec' - gem 'facter', "~> 2.4" - gem 'pry' - gem 'puppet-lint' - gem 'puppet-strings' - gem 'puppet-syntax' - gem 'puppetlabs_spec_helper', '>= 2.7.0' - gem 'rake' - gem 'rspec', '~> 3.0' - gem 'rspec-puppet', '~> 2.6' - gem 'rspec-puppet-facts' - gem 'rspec-puppet-utils' - gem 'rspec-retry' - # Required to test against Ruby 1.9 - gem 'rubocop', '~> 0.41.2' - gem 'rubysl-securerandom' - gem 'webmock' - - # Extra Puppet-lint gems - gem 'puppet-lint-appends-check', - :git => 'https://github.com/voxpupuli/puppet-lint-appends-check', - :ref => '07be8ce22d69353db055820b60bb77fe020238a6', - :require => false - gem 'puppet-lint-empty_string-check', :require => false - gem 'puppet-lint-file_ensure-check', :require => false - gem 'puppet-lint-leading_zero-check', :require => false - gem 'puppet-lint-param-docs', :require => false - gem 'puppet-lint-trailing_comma-check', :require => false - gem 'puppet-lint-undef_in_function-check', :require => false - gem 'puppet-lint-unquoted_string-check', :require => false - gem 'puppet-lint-version_comparison-check', :require => false + gem 'voxpupuli-test', '~> 5.0', :require => false + gem 'coveralls', :require => false + gem 'simplecov-console', :require => false + gem 'puppet_metadata', '~> 1.0', :require => false + gem 'bcrypt', :require => false + gem 'webmock', :require => false end group :development do - gem 'puppet-blacksmith' + gem 'guard-rake', :require => false + gem 'overcommit', '>= 0.39.1', :require => false end group :system_tests do - gem 'bcrypt' - gem 'beaker', '>= 4.2.0' - gem 'beaker-rspec', '~> 6.0' - gem 'beaker-docker' - gem 'beaker-puppet' - gem 'beaker-puppet_install_helper' - gem 'simp-beaker-helpers' - gem 'docker-api', '~> 1.0' - gem 'infrataster' - gem 'vault' + gem 'voxpupuli-acceptance', '~> 1.0', :require => false + gem 'bcrypt', :require => false + gem 'rspec-retry', :require => false + gem 'infrataster', :require => false +end + +group :release do + gem 'github_changelog_generator', '>= 1.16.1', :require => false if RUBY_VERSION >= '2.5' + gem 'voxpupuli-release', '>= 1.2.0', :require => false + gem 'puppet-strings', '>= 2.2', :require => false end + +gem 'rake', :require => false +gem 'facter', ENV['FACTER_GEM_VERSION'], :require => false, :groups => [:test] + +puppetversion = ENV['PUPPET_VERSION'] || '>= 6.0' +gem 'puppet', puppetversion, :require => false, :groups => [:test] + +# vim: syntax=ruby diff --git a/Rakefile b/Rakefile index 715ae18..f92f051 100644 --- a/Rakefile +++ b/Rakefile @@ -1,219 +1,72 @@ -require 'digest/sha1' -require 'rubygems' -require 'puppetlabs_spec_helper/rake_tasks' -require 'puppet_blacksmith/rake_tasks' -require 'net/http' -require 'uri' -require 'fileutils' -require 'rspec/core/rake_task' -require 'puppet-strings' -require 'puppet-strings/tasks' -require 'yaml' -require 'json' -require_relative 'spec/spec_utilities' - -ENV['VAULT_APPROLE_ROLE_ID'] ||= '48adc137-3270-fc4a-ae65-1306919d4bb0' -oss_package = ENV['OSS_PACKAGE'] and ENV['OSS_PACKAGE'] == 'true' - -elasticsearch_default_version = '7.10.1' - -# Workaround for certain rspec/beaker versions -module TempFixForRakeLastComment - def last_comment - last_description +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +# Attempt to load voxpupuli-test (which pulls in puppetlabs_spec_helper), +# otherwise attempt to load it directly. +begin + require 'voxpupuli/test/rake' +rescue LoadError + begin + require 'puppetlabs_spec_helper/rake_tasks' + rescue LoadError end end -Rake::Application.send :include, TempFixForRakeLastComment - -exclude_paths = [ - 'pkg/**/*', - 'vendor/**/*', - 'spec/**/*' -] - -require 'puppet-lint/tasks/puppet-lint' -require 'puppet-syntax/tasks/puppet-syntax' - -PuppetSyntax.exclude_paths = exclude_paths -PuppetSyntax.future_parser = true if ENV['FUTURE_PARSER'] == 'true' - -%w[ - 80chars - class_inherits_from_params_class - class_parameter_defaults - single_quote_string_with_variable -].each do |check| - PuppetLint.configuration.send("disable_#{check}") -end - -PuppetLint.configuration.ignore_paths = exclude_paths -PuppetLint.configuration.log_format = \ - '%{path}:%{line}:%{check}:%{KIND}:%{message}' - -# Append custom cleanup tasks to :clean -task :clean => [ - :'artifact:clean', - :spec_clean -] - -desc 'remove outdated module fixtures' -task :spec_prune do - mods = 'spec/fixtures/modules' - fixtures = YAML.load_file '.fixtures.yml' - fixtures['fixtures']['forge_modules'].each do |mod, params| - next unless params.is_a? Hash \ - and params.key? 'ref' \ - and File.exist? "#{mods}/#{mod}" - - metadata = JSON.parse(File.read("#{mods}/#{mod}/metadata.json")) - FileUtils.rm_rf "#{mods}/#{mod}" unless metadata['version'] == params['ref'] - end -end -task :spec_prep => [:spec_prune] - -RSpec::Core::RakeTask.new(:spec_verbose) do |t| - t.pattern = 'spec/{classes,defines,unit,functions,templates}/**/*_spec.rb' - t.rspec_opts = [ - '--format documentation', - '--require "ci/reporter/rspec"', - '--format CI::Reporter::RSpecFormatter', - '--color' - ] -end -task :spec_verbose => :spec_prep -RSpec::Core::RakeTask.new(:spec_puppet) do |t| - t.pattern = 'spec/{classes,defines,functions,templates,unit/facter}/**/*_spec.rb' - t.rspec_opts = ['--color'] +# load optional tasks for acceptance +# only available if gem group releases is installed +begin + require 'voxpupuli/acceptance/rake' +rescue LoadError end -task :spec_puppet => :spec_prep -RSpec::Core::RakeTask.new(:spec_unit) do |t| - t.pattern = 'spec/unit/{type,provider}/**/*_spec.rb' - t.rspec_opts = ['--color'] +# load optional tasks for releases +# only available if gem group releases is installed +begin + require 'voxpupuli/release/rake_tasks' +rescue LoadError end -task :spec_unit => :spec_prep - -task :beaker => [:spec_prep] - -desc 'Run all linting/unit tests.' -task :intake => [ - :syntax, - :rubocop, - :lint, - :validate, - :spec_unit, - :spec_puppet -] - -# Plumbing for snapshot tests -desc 'Run the snapshot tests' -RSpec::Core::RakeTask.new('beaker:snapshot', [:filter]) do |task, args| - task.rspec_opts = ['--color'] - task.pattern = 'spec/acceptance/tests/acceptance_spec.rb' - task.rspec_opts = [] - task.rspec_opts << '--format documentation' if ENV['CI'].nil? - task.rspec_opts << "--example '#{args[:filter]}'" if args[:filter] - ENV['SNAPSHOT_TEST'] = 'true' - if Rake::Task.task_defined? 'artifact:snapshot:not_found' - puts 'No snapshot artifacts found, skipping snapshot tests.' - exit(0) +desc "Run main 'test' task and report merged results to coveralls" +task test_with_coveralls: [:test] do + if Dir.exist?(File.expand_path('../lib', __FILE__)) + require 'coveralls/rake/task' + Coveralls::RakeTask.new + Rake::Task['coveralls:push'].invoke + else + puts 'Skipping reporting to coveralls. Module has no lib dir' end end -beaker_node_sets.each do |node| - desc "Run the snapshot tests against the #{node} nodeset" - task "beaker:#{node}:snapshot", [:filter] => %w[ - spec_prep - artifact:snapshot:deb - artifact:snapshot:rpm - ] do |_task, args| - ENV['BEAKER_set'] = node - Rake::Task['beaker:snapshot'].reenable - Rake::Task['beaker:snapshot'].invoke args[:filter] - end - - desc "Run acceptance tests against #{node}" - RSpec::Core::RakeTask.new( - "beaker:#{node}:acceptance", [:version, :filter] => [:spec_prep] - ) do |task, args| - ENV['BEAKER_set'] = node - args.with_defaults(:version => elasticsearch_default_version, :filter => nil) - task.pattern = 'spec/acceptance/tests/acceptance_spec.rb' - task.rspec_opts = [] - task.rspec_opts << '--format documentation' - task.rspec_opts << "--example '#{args[:filter]}'" if args[:filter] - ENV['ELASTICSEARCH_VERSION'] ||= args[:version] - Rake::Task['artifact:fetch'].invoke(ENV['ELASTICSEARCH_VERSION']) - end +desc 'Generate REFERENCE.md' +task :reference, [:debug, :backtrace] do |t, args| + patterns = '' + Rake::Task['strings:generate:reference'].invoke(patterns, args[:debug], args[:backtrace]) end -namespace :artifact do - desc 'Fetch specific installation artifacts' - task :fetch, [:version] do |_t, args| - fetch_archives( - derive_artifact_urls_for(args[:version]) - ) +begin + require 'github_changelog_generator/task' + require 'puppet_blacksmith' + GitHubChangelogGenerator::RakeTask.new :changelog do |config| + metadata = Blacksmith::Modulefile.new + config.future_release = "v#{metadata.version}" if metadata.version =~ /^\d+\.\d+.\d+$/ + config.header = "# Changelog\n\nAll notable changes to this project will be documented in this file.\nEach new release typically also includes the latest modulesync defaults.\nThese should not affect the functionality of the module." + config.exclude_labels = %w{duplicate question invalid wontfix wont-fix modulesync skip-changelog} + config.user = 'voxpupuli' + config.project = metadata.metadata['name'] end - namespace :snapshot do - snapshot_version = JSON.parse(http_retry('https://artifacts-api.elastic.co/v1/versions'))['versions'].reject do |version| - version.include? 'alpha' - end.last - - ENV['snapshot_version'] = snapshot_version - - downloads = JSON.parse(http_retry("https://artifacts-api.elastic.co/v1/search/#{snapshot_version}/elasticsearch"))['packages'].select do |pkg, _| - pkg =~ /(?:deb|rpm)/ and (oss_package ? pkg =~ /oss/ : pkg !~ /oss/) - end.map do |package, urls| - [package.split('.').last, urls] - end.to_h - - # We end up with something like: - # { - # 'rpm' => {'url' => 'https://...', 'sha_url' => 'https://...'}, - # 'deb' => {'url' => 'https://...', 'sha_url' => 'https://...'} - # } - # Note that checksums are currently broken on the Elastic unified release - # side; once they start working we can verify them. - - if downloads.empty? - puts 'No snapshot release available; skipping snapshot download' - %w[deb rpm].each { |ext| task ext } - task 'not_found' - else - # Download snapshot files - downloads.each_pair do |extension, urls| - filename = artifact urls['url'] - checksum = artifact urls['sha_url'] - link = artifact "elasticsearch-snapshot.#{extension}" - FileUtils.rm link if File.exist? link - - task extension => link - file link => filename do - unless File.exist?(link) and File.symlink?(link) \ - and File.readlink(link) == filename - File.delete link if File.exist? link - File.symlink File.basename(filename), link - end - end - - # file filename => checksum do - file filename do - get urls['url'], filename - end - - task checksum do - File.delete checksum if File.exist? checksum - get urls['sha_url'], checksum - end - end + # Workaround for https://github.com/github-changelog-generator/github-changelog-generator/issues/715 + require 'rbconfig' + if RbConfig::CONFIG['host_os'] =~ /linux/ + task :changelog do + puts 'Fixing line endings...' + changelog_file = File.join(__dir__, 'CHANGELOG.md') + changelog_txt = File.read(changelog_file) + new_contents = changelog_txt.gsub(%r{\r\n}, "\n") + File.open(changelog_file, "w") {|file| file.puts new_contents } end end - desc 'Purge fetched artifacts' - task :clean do - FileUtils.rm_rf(Dir.glob('spec/fixtures/artifacts/*')) - end +rescue LoadError end +# vim: syntax=ruby diff --git a/spec/acceptance/nodesets/centos-6-x64.yml b/spec/acceptance/nodesets/centos-6-x64.yml deleted file mode 100644 index 40b294b..0000000 --- a/spec/acceptance/nodesets/centos-6-x64.yml +++ /dev/null @@ -1,19 +0,0 @@ -HOSTS: - centos-6-x64: - roles: - - master - - agent - - database - - dashboard - platform: el-6-x86_64 - image: centos:6.9 - hypervisor: docker - docker_cmd: ["/sbin/init"] - docker_preserve_image: true - docker_image_commands: - - yum install -y wget tar which java-1.8.0-openjdk-headless - - rm /etc/init/tty.conf - - echo -e "elasticsearch hard nproc 4096\nelasticsearch soft nproc 4096" >> /etc/security/limits.conf - - echo -e "esuser hard nproc 4096\nesuser soft nproc 4096" >> /etc/security/limits.conf -CONFIG: - log_level: warn diff --git a/spec/acceptance/nodesets/centos-7-x64.yml b/spec/acceptance/nodesets/centos-7-x64.yml deleted file mode 100644 index dc95ee4..0000000 --- a/spec/acceptance/nodesets/centos-7-x64.yml +++ /dev/null @@ -1,20 +0,0 @@ -HOSTS: - centos-7-x64: - roles: - - agent - - master - - database - - dashboard - platform: el-7-x86_64 - image: centos:7 - hypervisor: docker - docker_cmd: ["/sbin/init"] - docker_preserve_image: true - docker_image_commands: - - yum install -y wget which cronie iproute - - mkdir -p /etc/selinux/targeted/contexts/ - - echo '' > /etc/selinux/targeted/contexts/dbus_contexts - - rm /lib/systemd/system/systemd*udev* - - rm /lib/systemd/system/getty.target -CONFIG: - log_level: warn diff --git a/spec/acceptance/nodesets/ubuntu-server-1404-x64.yml b/spec/acceptance/nodesets/ubuntu-server-1404-x64.yml deleted file mode 100644 index 6dea31b..0000000 --- a/spec/acceptance/nodesets/ubuntu-server-1404-x64.yml +++ /dev/null @@ -1,24 +0,0 @@ -HOSTS: - ubuntu-14-04: - roles: - - agent - - master - - database - - dashboard - platform: ubuntu-14.04-amd64 - image: ubuntu:14.04.5 - hypervisor: docker - docker_cmd: ["/sbin/init"] - docker_preserve_image: true - docker_image_commands: - - apt-get update - - apt-get install -yq apt-transport-https libssl-dev software-properties-common - - add-apt-repository -y ppa:openjdk-r/ppa - - apt-get update - - apt-get install -y openjdk-8-jre-headless - - update-ca-certificates -f - - ln -sf /sbin/initctl.distrib /sbin/initctl - - locale-gen en_US en_US.UTF-8 - - dpkg-reconfigure locales -CONFIG: - log_level: warn diff --git a/spec/acceptance/nodesets/ubuntu-server-1604-x64.yml b/spec/acceptance/nodesets/ubuntu-server-1604-x64.yml deleted file mode 100644 index 6a665dc..0000000 --- a/spec/acceptance/nodesets/ubuntu-server-1604-x64.yml +++ /dev/null @@ -1,17 +0,0 @@ -HOSTS: - ubuntu-16-04: - roles: - - agent - - master - - database - - dashboard - platform: ubuntu-16.04-amd64 - image: ubuntu:16.04 - hypervisor: docker - docker_cmd: ["/sbin/init"] - docker_preserve_image: true - docker_image_commands: - - apt-get update - - apt-get install -yq libssl-dev apt-transport-https -CONFIG: - log_level: warn diff --git a/spec/defines/004_elasticsearch_plugin_spec.rb b/spec/defines/004_elasticsearch_plugin_spec.rb index f22e321..2b66c3d 100644 --- a/spec/defines/004_elasticsearch_plugin_spec.rb +++ b/spec/defines/004_elasticsearch_plugin_spec.rb @@ -1,307 +1,308 @@ require 'spec_helper' +require 'helpers/class_shared_examples' describe 'elasticsearch::plugin', :type => 'define' do let(:title) { 'mobz/elasticsearch-head/1.0.0' } on_supported_os( :hardwaremodels => ['x86_64'], :supported_os => [ { 'operatingsystem' => 'CentOS', 'operatingsystemrelease' => ['6'] } ] ).each do |_os, facts| let(:facts) do facts.merge('scenario' => '', 'common' => '') end let(:pre_condition) do <<-EOS class { "elasticsearch": config => { "node" => { "name" => "test" } } } EOS end context 'default values' do context 'present' do let(:params) do { :ensure => 'present', :configdir => '/etc/elasticsearch' } end it { is_expected.to compile } end context 'absent' do let(:params) do { :ensure => 'absent' } end it { is_expected.to compile } end context 'configdir' do it { should contain_elasticsearch__plugin( 'mobz/elasticsearch-head/1.0.0' ).with_configdir('/etc/elasticsearch') } it { should contain_elasticsearch_plugin( 'mobz/elasticsearch-head/1.0.0' ).with_configdir('/etc/elasticsearch') } end end context 'with module_dir' do context 'add a plugin' do let(:params) do { :ensure => 'present', :module_dir => 'head' } end it { should contain_elasticsearch__plugin( 'mobz/elasticsearch-head/1.0.0' ) } it { should contain_elasticsearch_plugin( 'mobz/elasticsearch-head/1.0.0' ) } it { should contain_file( '/usr/share/elasticsearch/plugins/head' ).that_requires( 'Elasticsearch_plugin[mobz/elasticsearch-head/1.0.0]' ) } end context 'remove a plugin' do let(:params) do { :ensure => 'absent', :module_dir => 'head' } end it { should contain_elasticsearch__plugin( 'mobz/elasticsearch-head/1.0.0' ) } it { should contain_elasticsearch_plugin( 'mobz/elasticsearch-head/1.0.0' ).with( :ensure => 'absent' ) } it { should contain_file( '/usr/share/elasticsearch/plugins/head' ).that_requires( 'Elasticsearch_plugin[mobz/elasticsearch-head/1.0.0]' ) } end end context 'with url' do context 'add a plugin with full name' do let(:params) do { :ensure => 'present', :url => 'https://github.com/mobz/elasticsearch-head/archive/master.zip' } end it { should contain_elasticsearch__plugin('mobz/elasticsearch-head/1.0.0') } it { should contain_elasticsearch_plugin('mobz/elasticsearch-head/1.0.0').with(:ensure => 'present', :url => 'https://github.com/mobz/elasticsearch-head/archive/master.zip') } end end context 'offline plugin install' do let(:title) { 'head' } let(:params) do { :ensure => 'present', :source => 'puppet:///path/to/my/plugin.zip' } end it { should contain_elasticsearch__plugin('head') } it { should contain_file('/opt/elasticsearch/swdl/plugin.zip').with(:source => 'puppet:///path/to/my/plugin.zip', :before => 'Elasticsearch_plugin[head]') } it { should contain_elasticsearch_plugin('head').with(:ensure => 'present', :source => '/opt/elasticsearch/swdl/plugin.zip') } end describe 'service restarts' do let(:title) { 'head' } let(:params) do { :ensure => 'present', :module_dir => 'head' } end context 'restart_on_change set to false (default)' do let(:pre_condition) do <<-EOS class { "elasticsearch": } EOS end it { should_not contain_elasticsearch_plugin( 'head' ).that_notifies( 'Service[elasticsearch]' )} include_examples 'class', :sysv end context 'restart_on_change set to true' do let(:pre_condition) do <<-EOS class { "elasticsearch": restart_on_change => true, } EOS end it { should contain_elasticsearch_plugin( 'head' ).that_notifies( 'Service[elasticsearch]' )} include_examples('class') end context 'restart_plugin_change set to false (default)' do let(:pre_condition) do <<-EOS class { "elasticsearch": restart_plugin_change => false, } EOS end it { should_not contain_elasticsearch_plugin( 'head' ).that_notifies( 'Service[elasticsearch]' )} include_examples('class') end context 'restart_plugin_change set to true' do let(:pre_condition) do <<-EOS class { "elasticsearch": restart_plugin_change => true, } EOS end it { should contain_elasticsearch_plugin( 'head' ).that_notifies( 'Service[elasticsearch]' )} include_examples('class') end end describe 'proxy arguments' do let(:title) { 'head' } context 'unauthenticated' do context 'on define' do let(:params) do { :ensure => 'present', :proxy_host => 'es.local', :proxy_port => 8080 } end it { should contain_elasticsearch_plugin( 'head' ).with_proxy( 'http://es.local:8080' )} end context 'on main class' do let(:params) do { :ensure => 'present' } end let(:pre_condition) do <<-EOS class { 'elasticsearch': proxy_url => 'https://es.local:8080', } EOS end it { should contain_elasticsearch_plugin( 'head' ).with_proxy( 'https://es.local:8080' )} end end context 'authenticated' do context 'on define' do let(:params) do { :ensure => 'present', :proxy_host => 'es.local', :proxy_port => 8080, :proxy_username => 'elastic', :proxy_password => 'password' } end it { should contain_elasticsearch_plugin( 'head' ).with_proxy( 'http://elastic:password@es.local:8080' )} end context 'on main class' do let(:params) do { :ensure => 'present' } end let(:pre_condition) do <<-EOS class { 'elasticsearch': proxy_url => 'http://elastic:password@es.local:8080', } EOS end it { should contain_elasticsearch_plugin( 'head' ).with_proxy( 'http://elastic:password@es.local:8080' )} end end end describe 'collector ordering' do describe 'present' do let(:title) { 'head' } let(:pre_condition) do <<-EOS class { 'elasticsearch': } EOS end it { should contain_elasticsearch__plugin( 'head' ).that_requires( 'Class[elasticsearch::config]' )} it { should contain_elasticsearch_plugin( 'head' ).that_comes_before( 'Service[elasticsearch]' )} include_examples 'class' end end end end diff --git a/spec/defines/007_elasticsearch_user_spec.rb b/spec/defines/007_elasticsearch_user_spec.rb index c8b3103..d96496b 100644 --- a/spec/defines/007_elasticsearch_user_spec.rb +++ b/spec/defines/007_elasticsearch_user_spec.rb @@ -1,120 +1,121 @@ require 'spec_helper' +require 'helpers/class_shared_examples' describe 'elasticsearch::user' do let(:title) { 'elastic' } let(:pre_condition) do <<-EOS class { 'elasticsearch': } EOS end on_supported_os( :hardwaremodels => ['x86_64'], :supported_os => [ { 'operatingsystem' => 'CentOS', 'operatingsystemrelease' => ['7'] } ] ).each do |os, facts| context "on #{os}" do let(:facts) { facts.merge( :scenario => '', :common => '' ) } context 'with default parameters' do let(:params) do { :password => 'foobar', :roles => %w[monitor user] } end it { should contain_elasticsearch__user('elastic') } it { should contain_elasticsearch_user('elastic') } it do should contain_elasticsearch_user_roles('elastic').with( 'ensure' => 'present', 'roles' => %w[monitor user] ) end end describe 'collector ordering' do describe 'when present' do let(:pre_condition) do <<-EOS class { 'elasticsearch': } elasticsearch::template { 'foo': content => {"foo" => "bar"} } elasticsearch::role { 'test_role': privileges => { 'cluster' => 'monitor', 'indices' => { '*' => 'all', }, }, } EOS end let(:params) do { :password => 'foobar', :roles => %w[monitor user] } end it { should contain_elasticsearch__role('test_role') } it { should contain_elasticsearch_role('test_role') } it { should contain_elasticsearch_role_mapping('test_role') } it { should contain_elasticsearch__user('elastic') .that_comes_before([ 'Elasticsearch::Template[foo]' ]).that_requires([ 'Elasticsearch::Role[test_role]' ])} include_examples 'class', :systemd end describe 'when absent' do let(:pre_condition) do <<-EOS class { 'elasticsearch': } elasticsearch::template { 'foo': content => {"foo" => "bar"} } elasticsearch::role { 'test_role': privileges => { 'cluster' => 'monitor', 'indices' => { '*' => 'all', }, }, } EOS end let(:params) do { :password => 'foobar', :roles => %w[monitor user] } end it { should contain_elasticsearch__role('test_role') } it { should contain_elasticsearch_role('test_role') } it { should contain_elasticsearch_role_mapping('test_role') } it { should contain_elasticsearch__user('elastic') .that_comes_before([ 'Elasticsearch::Template[foo]' ]).that_requires([ 'Elasticsearch::Role[test_role]' ])} include_examples 'class', :systemd end end end end end diff --git a/spec/defines/008_elasticsearch_role_spec.rb b/spec/defines/008_elasticsearch_role_spec.rb index df1e6b3..5804c50 100644 --- a/spec/defines/008_elasticsearch_role_spec.rb +++ b/spec/defines/008_elasticsearch_role_spec.rb @@ -1,109 +1,110 @@ require 'spec_helper' +require 'helpers/class_shared_examples' describe 'elasticsearch::role' do let(:title) { 'elastic_role' } let(:pre_condition) do <<-EOS class { 'elasticsearch': } EOS end let(:params) do { :privileges => { 'cluster' => '*' }, :mappings => [ 'cn=users,dc=example,dc=com', 'cn=admins,dc=example,dc=com', 'cn=John Doe,cn=other users,dc=example,dc=com' ] } end on_supported_os( :hardwaremodels => ['x86_64'], :supported_os => [ { 'operatingsystem' => 'CentOS', 'operatingsystemrelease' => ['7'] } ] ).each do |os, facts| context "on #{os}" do let(:facts) { facts.merge( :scenario => '', :common => '' ) } context 'with an invalid role name' do context 'too long' do let(:title) { 'A' * 41 } it { should raise_error(Puppet::Error, /expected length/i) } end end context 'with default parameters' do it { should contain_elasticsearch__role('elastic_role') } it { should contain_elasticsearch_role('elastic_role') } it do should contain_elasticsearch_role_mapping('elastic_role').with( 'ensure' => 'present', 'mappings' => [ 'cn=users,dc=example,dc=com', 'cn=admins,dc=example,dc=com', 'cn=John Doe,cn=other users,dc=example,dc=com' ] ) end end describe 'collector ordering' do describe 'when present' do let(:pre_condition) do <<-EOS class { 'elasticsearch': } elasticsearch::template { 'foo': content => {"foo" => "bar"} } elasticsearch::user { 'elastic': password => 'foobar', roles => ['elastic_role'], } EOS end it { should contain_elasticsearch__role('elastic_role') .that_comes_before([ 'Elasticsearch::Template[foo]', 'Elasticsearch::User[elastic]' ])} include_examples 'class', :systemd end describe 'when absent' do let(:pre_condition) do <<-EOS class { 'elasticsearch': } elasticsearch::template { 'foo': content => {"foo" => "bar"} } elasticsearch::user { 'elastic': password => 'foobar', roles => ['elastic_role'], } EOS end include_examples 'class', :systemd # TODO: Uncomment once upstream issue is fixed. # https://github.com/rodjek/rspec-puppet/issues/418 # it { should contain_elasticsearch__shield__role('elastic_role') # .that_comes_before([ # 'Elasticsearch::Template[foo]', # 'Elasticsearch::Plugin[shield]', # 'Elasticsearch::Shield::User[elastic]' # ])} end end end end end diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index cd90602..6bc659f 100644 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -1,21 +1,21 @@ -require_relative 'helpers/class_shared_examples' -require 'rspec-puppet-utils' -require 'rspec-puppet-facts' -include RspecPuppetFacts +# frozen_string_literal: true -def fixture_path - File.expand_path(File.join(__FILE__, '..', 'fixtures')) -end +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ + +# puppetlabs_spec_helper will set up coverage if the env variable is set. +# We want to do this if lib exists and it hasn't been explicitly set. +ENV['COVERAGE'] ||= 'yes' if Dir.exist?(File.expand_path('../lib', __dir__)) -$LOAD_PATH.unshift(File.expand_path(File.dirname(__FILE__) + '/../')) +require 'voxpupuli/test/spec_helper' RSpec.configure do |c| - c.mock_with :rspec + c.hiera_config = 'spec/fixtures/hiera/hiera.yaml' end -require 'puppetlabs_spec_helper/module_spec_helper' -RSpec.configure do |c| - c.add_setting :fixture_path, :default => fixture_path - # c.mock_with(:rspec) - c.hiera_config = File.join(fixture_path, '/hiera/hiera.yaml') +if File.exist?(File.join(__dir__, 'default_module_facts.yml')) + facts = YAML.safe_load(File.read(File.join(__dir__, 'default_module_facts.yml'))) + facts&.each do |name, value| + add_custom_fact name.to_sym, value + end end diff --git a/spec/spec_helper_acceptance.rb b/spec/spec_helper_acceptance.rb index 8431fcc..d3a6e23 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/spec_helper_acceptance.rb @@ -1,260 +1,10 @@ -require 'beaker-rspec' -require 'beaker/puppet_install_helper' -require 'securerandom' -require 'thread' -require 'infrataster/rspec' -require 'rspec/retry' -require 'vault' +# frozen_string_literal: true -require 'simp/beaker_helpers' -include Simp::BeakerHelpers +# Managed by modulesync - DO NOT EDIT +# https://voxpupuli.org/docs/updating-files-managed-with-modulesync/ -require_relative 'spec_helper_tls' -require_relative 'spec_utilities' -require_relative '../lib/puppet_x/elastic/deep_to_i' -require_relative '../lib/puppet_x/elastic/deep_to_s' +require 'voxpupuli/acceptance/spec_helper_acceptance' -# def f -# RSpec.configuration.fact -# end +configure_beaker -run_puppet_install_helper('agent') unless ENV['BEAKER_provision'] == 'no' - -RSpec.configure do |c| - # General-purpose spec-global variables - c.add_setting :v, :default => {} - - # Puppet debug logging - v[:puppet_debug] = ENV['BEAKER_debug'] ? true : false - - unless ENV['snapshot_version'].nil? - v[:snapshot_version] = ENV['snapshot_version'] - v[:is_snapshot] = ENV['SNAPSHOT_TEST'] == 'true' - end - - unless ENV['ELASTICSEARCH_VERSION'].nil? and v[:snapshot_version].nil? - v[:elasticsearch_full_version] = ENV['ELASTICSEARCH_VERSION'] || v[:snapshot_version] - v[:elasticsearch_major_version] = v[:elasticsearch_full_version].split('.').first.to_i - v[:elasticsearch_package] = {} - v[:template] = if v[:elasticsearch_major_version] == 6 - JSON.load(File.new('spec/fixtures/templates/6.x.json')) - elsif v[:elasticsearch_major_version] >= 8 - JSON.load(File.new('spec/fixtures/templates/post_8.0.json')) - else - JSON.load(File.new('spec/fixtures/templates/7.x.json')) - end - v[:template] = Puppet_X::Elastic.deep_to_i(Puppet_X::Elastic.deep_to_s(v[:template])) - v[:pipeline] = JSON.load(File.new('spec/fixtures/pipelines/example.json')) - - v[:elasticsearch_plugins] = Dir[ - artifact("*#{v[:elasticsearch_full_version]}.zip", ['plugins']) - ].map do |plugin| - plugin_filename = File.basename(plugin) - plugin_name = plugin_filename.match(/^(?.+)-#{v[:elasticsearch_full_version]}.zip/)[:name] - [ - plugin_name, - { - :path => plugin, - :url => derive_plugin_urls_for(v[:elasticsearch_full_version], [plugin_name]).keys.first - } - ] - end.to_h - end - - v[:oss] = (not ENV['OSS_PACKAGE'].nil?) and ENV['OSS_PACKAGE'] == 'true' - v[:cluster_name] = SecureRandom.hex(10) - - # rspec-retry - c.display_try_failure_messages = true - c.default_sleep_interval = 10 - # General-case retry keyword for unstable tests - c.around :each, :with_retries do |example| - example.run_with_retry retry: 10 - end - - # Helper hook for module cleanup - c.after :context, :with_cleanup do - apply_manifest <<-EOS - class { 'elasticsearch': - ensure => 'absent', - manage_repo => true, - oss => #{v[:oss]}, - } - - file { '/usr/share/elasticsearch/plugin': - ensure => 'absent', - force => true, - recurse => true, - require => Class['elasticsearch'], - } - EOS - end - - c.before :context, :with_certificates do - @keystore_password = SecureRandom.hex - @role = [*('a'..'z')].sample(8).join - - # Setup TLS cert placement - @tls = gen_certs(2, '/tmp') - - create_remote_file hosts, @tls[:ca][:cert][:path], @tls[:ca][:cert][:pem] - @tls[:clients].each do |node| - node.each do |_type, params| - create_remote_file hosts, params[:path], params[:pem] - end - end - end - - c.before :context, :with_license do - Vault.address = ENV['VAULT_ADDR'] - if ENV['CI'] - Vault.auth.approle(ENV['VAULT_APPROLE_ROLE_ID'], ENV['VAULT_APPROLE_SECRET_ID']) - else - Vault.auth.token(ENV['VAULT_TOKEN']) - end - licenses = Vault.with_retries(Vault::HTTPConnectionError) do - Vault.logical.read(ENV['VAULT_PATH']) - end.data - - raise 'No license found!' unless licenses - - # license = case v[:elasticsearch_major_version] - # when 6 - # licenses[:v5] - # else - # licenses[:v7] - # end - license = licenses[:v7] - create_remote_file hosts, '/tmp/license.json', license - v[:elasticsearch_license_path] = '/tmp/license.json' - end - - c.after :context, :then_purge do - shell 'rm -rf {/usr/share,/etc,/var/lib}/elasticsearch*' - end - - c.before :context, :first_purge do - shell 'rm -rf {/usr/share,/etc,/var/lib}/elasticsearch*' - end - - # Provide a hook filter to spit out some ES logs if the example fails. - c.after(:example, :logs_on_failure) do |example| - if example.exception - hosts.each do |host| - on host, "find / -name '#{v[:cluster_name]}.log' | xargs cat || true" do |result| - puts result.formatted_output - end - end - end - end -end - -files_dir = ENV['files_dir'] || './spec/fixtures/artifacts' - -# General bootstrapping steps for each host -hosts.each do |host| - # # Set the host to 'aio' in order to adopt the puppet-agent style of - # # installation, and configure paths/etc. - # host[:type] = 'aio' - # configure_defaults_on host, 'aio' - - if fact('os.family') == 'Suse' - install_package host, - '--force-resolution augeas-devel libxml2-devel ruby-devel' - on host, 'gem install ruby-augeas --no-ri --no-rdoc' - end - - v[:ext] = case fact('os.family') - when 'Debian' - 'deb' - else - 'rpm' - end - - if v[:elasticsearch_package] - v[:elasticsearch_package].merge!( - derive_full_package_url( - v[:elasticsearch_full_version], [v[:ext]] - ).flat_map do |url, filename| - [[:url, url], [:filename, filename], [:path, artifact(filename)]] - end.to_h - ) - end - - Infrataster::Server.define(:docker) do |server| - server.address = host[:ip] - server.ssh = host[:ssh].tap { |s| s.delete :forward_agent } - end - Infrataster::Server.define(:container) do |server| - server.address = host[:vm_ip] # this gets ignored anyway - server.from = :docker - end -end - -RSpec.configure do |c| - if v[:is_snapshot] - c.before :suite do - scp_to default, - "#{files_dir}/elasticsearch-snapshot.#{v[:ext]}", - "/tmp/elasticsearch-snapshot.#{v[:ext]}" - v[:snapshot_package] = "file:/tmp/elasticsearch-snapshot.#{v[:ext]}" - end - end - - c.before :suite do - # Install module and dependencies - install_dev_puppet_module :ignore_list => [ - 'junit' - ] + Beaker::DSL::InstallUtils::ModuleUtils::PUPPET_MODULE_INSTALL_IGNORE - - hosts.each do |host| - modules = %w[archive augeas_core datacat java java_ks stdlib elastic_stack] - - dist_module = { - 'Debian' => ['apt'], - 'Suse' => ['zypprepo'], - 'RedHat' => %w[concat yumrepo_core] - }[fact('os.family')] - - modules += dist_module unless dist_module.nil? - - modules.each do |mod| - copy_module_to( - host, - :module_name => mod, - :source => "spec/fixtures/modules/#{mod}" - ) - end - - on(host, 'mkdir -p etc/puppet/modules/another/files/') - - # Apt doesn't update package caches sometimes, ensure we're caught up. - shell 'apt-get update' if fact('os.family') == 'Debian' - end - - # Use the Java class once before the suite of tests - unless shell('command -v java', :accept_all_exit_codes => true).exit_code.zero? - java = case fact('os.name') - when 'OpenSuSE' - 'package => "java-1_8_0-openjdk-headless",' - else - '' - end - - apply_manifest <<-MANIFEST - class { "java" : - distribution => "jdk", - #{java} - } - MANIFEST - end - end -end - -# # Java 8 is only easy to manage on recent distros -# def v5x_capable? -# (fact('os.family') == 'RedHat' and \ -# not (fact('os.name') == 'OracleLinux' and \ -# f['os']['release']['major'] == '6')) or \ -# f.dig 'os', 'distro', 'codename' == 'xenial' -# end +Dir['./spec/support/acceptance/**/*.rb'].sort.each { |f| require f } diff --git a/spec/spec_helper_acceptance.rb b/spec/support/acceptance/elastic.rb similarity index 67% copy from spec/spec_helper_acceptance.rb copy to spec/support/acceptance/elastic.rb index 8431fcc..b12f969 100644 --- a/spec/spec_helper_acceptance.rb +++ b/spec/support/acceptance/elastic.rb @@ -1,260 +1,221 @@ -require 'beaker-rspec' -require 'beaker/puppet_install_helper' -require 'securerandom' -require 'thread' +# frozen_string_literal: true + require 'infrataster/rspec' +require 'securerandom' require 'rspec/retry' -require 'vault' -require 'simp/beaker_helpers' -include Simp::BeakerHelpers - -require_relative 'spec_helper_tls' -require_relative 'spec_utilities' -require_relative '../lib/puppet_x/elastic/deep_to_i' -require_relative '../lib/puppet_x/elastic/deep_to_s' +require_relative '../../spec_helper_tls' +require_relative '../../spec_utilities' +require_relative '../../../lib/puppet_x/elastic/deep_to_i' +require_relative '../../../lib/puppet_x/elastic/deep_to_s' # def f # RSpec.configuration.fact # end run_puppet_install_helper('agent') unless ENV['BEAKER_provision'] == 'no' RSpec.configure do |c| # General-purpose spec-global variables - c.add_setting :v, :default => {} + c.add_setting :v, default: {} # Puppet debug logging v[:puppet_debug] = ENV['BEAKER_debug'] ? true : false unless ENV['snapshot_version'].nil? v[:snapshot_version] = ENV['snapshot_version'] v[:is_snapshot] = ENV['SNAPSHOT_TEST'] == 'true' end - unless ENV['ELASTICSEARCH_VERSION'].nil? and v[:snapshot_version].nil? + unless ENV['ELASTICSEARCH_VERSION'].nil? && v[:snapshot_version].nil? v[:elasticsearch_full_version] = ENV['ELASTICSEARCH_VERSION'] || v[:snapshot_version] v[:elasticsearch_major_version] = v[:elasticsearch_full_version].split('.').first.to_i v[:elasticsearch_package] = {} v[:template] = if v[:elasticsearch_major_version] == 6 - JSON.load(File.new('spec/fixtures/templates/6.x.json')) + JSON.parse(File.read('spec/fixtures/templates/6.x.json')) elsif v[:elasticsearch_major_version] >= 8 - JSON.load(File.new('spec/fixtures/templates/post_8.0.json')) + JSON.parse(File.read('spec/fixtures/templates/post_8.0.json')) else - JSON.load(File.new('spec/fixtures/templates/7.x.json')) + JSON.parse(File.read('spec/fixtures/templates/7.x.json')) end v[:template] = Puppet_X::Elastic.deep_to_i(Puppet_X::Elastic.deep_to_s(v[:template])) - v[:pipeline] = JSON.load(File.new('spec/fixtures/pipelines/example.json')) - - v[:elasticsearch_plugins] = Dir[ - artifact("*#{v[:elasticsearch_full_version]}.zip", ['plugins']) - ].map do |plugin| - plugin_filename = File.basename(plugin) - plugin_name = plugin_filename.match(/^(?.+)-#{v[:elasticsearch_full_version]}.zip/)[:name] - [ - plugin_name, - { - :path => plugin, - :url => derive_plugin_urls_for(v[:elasticsearch_full_version], [plugin_name]).keys.first - } - ] - end.to_h + v[:pipeline] = JSON.parse(File.read('spec/fixtures/pipelines/example.json')) end - v[:oss] = (not ENV['OSS_PACKAGE'].nil?) and ENV['OSS_PACKAGE'] == 'true' + v[:elasticsearch_plugins] = Dir[ + artifact("*#{v[:elasticsearch_full_version]}.zip", ['plugins']) + ].map do |plugin| + plugin_filename = File.basename(plugin) + plugin_name = plugin_filename.match(%r{^(?.+)-#{v[:elasticsearch_full_version]}.zip})[:name] + [ + plugin_name, + { + path: plugin, + url: derive_plugin_urls_for(v[:elasticsearch_full_version], [plugin_name]).keys.first, + }, + ] + end.to_h + + v[:oss] = !ENV['OSS_PACKAGE'].nil? and ENV['OSS_PACKAGE'] == 'true' v[:cluster_name] = SecureRandom.hex(10) # rspec-retry c.display_try_failure_messages = true c.default_sleep_interval = 10 # General-case retry keyword for unstable tests c.around :each, :with_retries do |example| example.run_with_retry retry: 10 end # Helper hook for module cleanup c.after :context, :with_cleanup do - apply_manifest <<-EOS + apply_manifest <<-MANIFEST class { 'elasticsearch': ensure => 'absent', manage_repo => true, oss => #{v[:oss]}, } - file { '/usr/share/elasticsearch/plugin': ensure => 'absent', force => true, recurse => true, require => Class['elasticsearch'], } - EOS + MANIFEST end c.before :context, :with_certificates do @keystore_password = SecureRandom.hex @role = [*('a'..'z')].sample(8).join # Setup TLS cert placement @tls = gen_certs(2, '/tmp') create_remote_file hosts, @tls[:ca][:cert][:path], @tls[:ca][:cert][:pem] @tls[:clients].each do |node| node.each do |_type, params| create_remote_file hosts, params[:path], params[:pem] end end end c.before :context, :with_license do Vault.address = ENV['VAULT_ADDR'] if ENV['CI'] Vault.auth.approle(ENV['VAULT_APPROLE_ROLE_ID'], ENV['VAULT_APPROLE_SECRET_ID']) else Vault.auth.token(ENV['VAULT_TOKEN']) end licenses = Vault.with_retries(Vault::HTTPConnectionError) do Vault.logical.read(ENV['VAULT_PATH']) end.data raise 'No license found!' unless licenses # license = case v[:elasticsearch_major_version] # when 6 # licenses[:v5] # else # licenses[:v7] # end license = licenses[:v7] create_remote_file hosts, '/tmp/license.json', license v[:elasticsearch_license_path] = '/tmp/license.json' end c.after :context, :then_purge do shell 'rm -rf {/usr/share,/etc,/var/lib}/elasticsearch*' end c.before :context, :first_purge do shell 'rm -rf {/usr/share,/etc,/var/lib}/elasticsearch*' end # Provide a hook filter to spit out some ES logs if the example fails. c.after(:example, :logs_on_failure) do |example| if example.exception hosts.each do |host| on host, "find / -name '#{v[:cluster_name]}.log' | xargs cat || true" do |result| puts result.formatted_output end end end end end files_dir = ENV['files_dir'] || './spec/fixtures/artifacts' # General bootstrapping steps for each host hosts.each do |host| # # Set the host to 'aio' in order to adopt the puppet-agent style of # # installation, and configure paths/etc. # host[:type] = 'aio' # configure_defaults_on host, 'aio' if fact('os.family') == 'Suse' install_package host, '--force-resolution augeas-devel libxml2-devel ruby-devel' on host, 'gem install ruby-augeas --no-ri --no-rdoc' end v[:ext] = case fact('os.family') when 'Debian' 'deb' else 'rpm' end - if v[:elasticsearch_package] - v[:elasticsearch_package].merge!( - derive_full_package_url( - v[:elasticsearch_full_version], [v[:ext]] - ).flat_map do |url, filename| - [[:url, url], [:filename, filename], [:path, artifact(filename)]] - end.to_h - ) - end + v[:elasticsearch_package]&.merge!( + derive_full_package_url( + v[:elasticsearch_full_version], [v[:ext]] + ).flat_map do |url, filename| + [[:url, url], [:filename, filename], [:path, artifact(filename)]] + end.to_h + ) Infrataster::Server.define(:docker) do |server| server.address = host[:ip] server.ssh = host[:ssh].tap { |s| s.delete :forward_agent } end Infrataster::Server.define(:container) do |server| server.address = host[:vm_ip] # this gets ignored anyway server.from = :docker end end RSpec.configure do |c| if v[:is_snapshot] c.before :suite do scp_to default, "#{files_dir}/elasticsearch-snapshot.#{v[:ext]}", "/tmp/elasticsearch-snapshot.#{v[:ext]}" v[:snapshot_package] = "file:/tmp/elasticsearch-snapshot.#{v[:ext]}" end end c.before :suite do - # Install module and dependencies - install_dev_puppet_module :ignore_list => [ - 'junit' - ] + Beaker::DSL::InstallUtils::ModuleUtils::PUPPET_MODULE_INSTALL_IGNORE - - hosts.each do |host| - modules = %w[archive augeas_core datacat java java_ks stdlib elastic_stack] - - dist_module = { - 'Debian' => ['apt'], - 'Suse' => ['zypprepo'], - 'RedHat' => %w[concat yumrepo_core] - }[fact('os.family')] - - modules += dist_module unless dist_module.nil? - - modules.each do |mod| - copy_module_to( - host, - :module_name => mod, - :source => "spec/fixtures/modules/#{mod}" - ) - end - - on(host, 'mkdir -p etc/puppet/modules/another/files/') - - # Apt doesn't update package caches sometimes, ensure we're caught up. - shell 'apt-get update' if fact('os.family') == 'Debian' - end - # Use the Java class once before the suite of tests - unless shell('command -v java', :accept_all_exit_codes => true).exit_code.zero? + unless shell('command -v java', accept_all_exit_codes: true).exit_code.zero? java = case fact('os.name') when 'OpenSuSE' 'package => "java-1_8_0-openjdk-headless",' else '' end apply_manifest <<-MANIFEST class { "java" : distribution => "jdk", #{java} } MANIFEST end end end - # # Java 8 is only easy to manage on recent distros # def v5x_capable? # (fact('os.family') == 'RedHat' and \ # not (fact('os.name') == 'OracleLinux' and \ # f['os']['release']['major'] == '6')) or \ # f.dig 'os', 'distro', 'codename' == 'xenial' # end diff --git a/spec/unit/facter/es_facts_spec.rb b/spec/unit/facter/es_facts_spec.rb index d6bfe9e..b5d208c 100644 --- a/spec/unit/facter/es_facts_spec.rb +++ b/spec/unit/facter/es_facts_spec.rb @@ -1,107 +1,113 @@ require 'spec_helper' +require 'spec_utilities' + require 'webmock/rspec' +def fixture_path + File.expand_path(File.join(__dir__, '..', '..', 'fixtures')) +end + describe 'elasticsearch facts' do before(:each) do stub_request(:get, 'http://localhost:9200/') .with(:headers => { 'Accept' => '*/*', 'User-Agent' => 'Ruby' }) .to_return( :status => 200, :body => File.read( File.join( fixture_path, 'facts/Warlock-root.json' ) ) ) stub_request(:get, 'http://localhost:9200/_nodes/Warlock') .with(:headers => { 'Accept' => '*/*', 'User-Agent' => 'Ruby' }) .to_return( :status => 200, :body => File.read( File.join( fixture_path, 'facts/Warlock-nodes.json' ) ) ) allow(File) .to receive(:directory?) .and_return(true) allow(File) .to receive(:readable?) .and_return(true) allow(YAML) .to receive(:load_file) .with('/etc/elasticsearch/elasticsearch.yml', any_args) .and_return({}) - require 'lib/facter/es_facts' + require_relative '../../../lib/facter/es_facts' end describe 'elasticsearch_port' do it 'finds listening port' do expect(Facter.fact(:elasticsearch_port).value) .to eq('9200') end end describe 'instance' do it 'returns the node name' do expect(Facter.fact(:elasticsearch_name).value).to eq('Warlock') end it 'returns the node version' do expect(Facter.fact(:elasticsearch_version).value).to eq('1.4.2') end it 'returns the cluster name' do expect(Facter.fact(:elasticsearch_cluster_name).value) .to eq('elasticsearch') end it 'returns the node ID' do expect(Facter.fact(:elasticsearch_node_id).value) .to eq('yQAWBO3FS8CupZnSvAVziQ') end it 'returns the mlockall boolean' do expect(Facter.fact(:elasticsearch_mlockall).value).to be_falsy end it 'returns installed plugins' do expect(Facter.fact(:elasticsearch_plugins).value).to eq('kopf') end describe 'kopf plugin' do it 'returns the correct version' do expect(Facter.fact(:elasticsearch_plugin_kopf_version).value) .to eq('1.4.3') end it 'returns the correct description' do expect(Facter.fact(:elasticsearch_plugin_kopf_description).value) .to eq('kopf - simple web administration tool for ElasticSearch') end it 'returns the plugin URL' do expect(Facter.fact(:elasticsearch_plugin_kopf_url).value) .to eq('/_plugin/kopf/') end it 'returns the plugin JVM boolean' do expect(Facter.fact(:elasticsearch_plugin_kopf_jvm).value) .to be_falsy end it 'returns the plugin _site boolean' do expect(Facter.fact(:elasticsearch_plugin_kopf_site).value) .to be_truthy end end # of describe plugin end # of describe instance end # of describe elasticsearch facts