diff --git a/README.md b/README.md index 6ee0baf..013593e 100644 --- a/README.md +++ b/README.md @@ -1,140 +1,154 @@ # puppet-varnish [![Build Status](https://secure.travis-ci.org/claranet/puppet-varnish.png?branch=master)](http://travis-ci.org/claranet/puppet-varnish) [![Puppet Forge](http://img.shields.io/puppetforge/v/claranet/varnish.svg)](https://forge.puppetlabs.com/claranet/varnish) [![Forge Downloads](https://img.shields.io/puppetforge/dt/claranet/varnish.svg)](https://forge.puppetlabs.com/claranet/varnish) #### Table of Contents 1. [Overview - What is the puppet-varnish module?](#overview) 1. [Module Description - What does the module do?](#module-description) 1. [Setup - The basics of getting started with puppet-varnish](#setup) 1. [Usage - Configuration options and additional functionality](#usage) * [Examples](#examples) * [Parameter Reference](#parameter-reference) 1. [Limitations - OS compatibility, etc.](#limitations) 1. [Development - Guide for contributing to the module](#development) ## Overview This module Installs and configures Varnish. ## Module Description This module Supports Varnish versions 3.0, 4.0, 4.1, 5.0, 5.1 and 5.2 across Ubuntu 14.04/16.04, Debian 7/8 and RedHat derivates 6/7. This module will install Varnish, **by default version 4.1** from the official Packagecloud repositories, adding EPEL for RedHat-like systems and working around a SELinux policy bug in RHEL/CentOS 6 for Varnish 4.0 and above. It will also install and configure a Systemd service for certain OS/Varnish combinations. If necessary, you can specify any of the Varnish versions above, although there are imcompatibilities with some versions of Varnish and some OS versions, see [Limitations](#limitations). ## Setup To accept all default parameters - at minimum it is suggested you set a secret (if not explicitly set, one will be created via `/proc/sys/kernel/random/uuid`) and overwrite the packaged `default.vcl`. ```puppet class { '::varnish': secret => '6565bd1c-b6d1-4ba3-99bc-3c7a41ffd94f', } ::varnish::vcl { '/etc/varnish/default.vcl': content => template('data/varnish/default.vcl.erb'), } ``` ### Multiple Listen Interfaces Varnish supports listening on multiple interfaces. The module implements this by exposing a `listen` parameter, which can either be set to a String value for one interface (e.g. `127.0.0.1` or `0.0.0.0`), or an array of values. By default, the the module will append `listen_port` to each element of the array - however to set a different port for each interface, just append it using standard notation, for example: `127.0.0.1:8080`. ## Usage ### Examples To use a static file with `varnish::vcl` rather than a template: ```puppet ::varnish::vcl { '/etc/varnish/default.vcl': content => file('data/varnish/default.vcl'), # Equivalent to: source => 'puppet:///modules/data/varnish/default.vcl' } ``` To pin Varnish to a specific version - you may also provide `varnish_version` as long as it matches the major and minor version in `package_ensure`, however the module will automatically calculate `varnish_version` if not set: ```puppet class { '::varnish': package_ensure => '4.0.5-1~xenial', } ``` To configure Varnish to listen on port 8080 on localhost and port 6081 on `172.16.100.10`: ```puppet class { '::varnish': listen => ['127.0.0.1:8080','172.16.100.10:6081'], } ``` To configure Varnish to listen on port 80, specifically on localhost and `192.168.1.195`: ```puppet class { '::varnish': listen => ['127.0.0.1','192.168.1.195'], listen_port => '80', } ``` +To use multiple storage backends in varnish for example a primary `4GB memory backend` and a `50GB file backend`: + +```puppet + class { '::varnish': + storage_type => 'malloc', + storage_size => '4G', + storage_additional => [ + 'file,/var/lib/varnish/varnish_additional.bin,50G', + + ] + } +``` + ### Parameter Reference |Parameter|Description| |---------|-----------| |addrepo|Whether to add the official Varnish repos| |varnish_version|Major Varnish version| |secret|Secret for admin access| |secret_file|File to store the secret| |vcl_conf|Varnish vcl config file path| |listen|IP to bind to| |listen_port|TCP port to listen on| |admin_listen|Admin IP to bind to| |admin_port|TCP port for admin to listen on| |min_threads|Minimum Varnish worker threads| |max_threads|Maximum Varnish worker threads| |thread_timeout|Terminate threads after this long idle| |storage_type|malloc or file| |storage_file|File to mmap on disk for cache storage| |storage_size|Size of storage file or RAM, eg 10G or 50%| +|storage_additional|Hash of additional storage backends, passed plainly to varnishd -s after the normal configured storage backends| |package_ensure|Version of Varnish package to install, eg 3.0.5-1.el6| |runtime_params|hash of run-time parameters to be specified at startup| ## Limitations There are several limitations with various Varnish and OS combinations. The module will attempt to flag known issues, however: * Varnish 3.0 is not supported on Ubuntu 16.04 * Varnish 5.0 supports **only** Debian 8 and Ubuntu 16.04 ## Development * Copyright (C) 2017 Claranet * Distributed under the terms of the Apache License v2.0 - see LICENSE file for details. diff --git a/manifests/init.pp b/manifests/init.pp index a2e6139..07008b9 100644 --- a/manifests/init.pp +++ b/manifests/init.pp @@ -1,121 +1,124 @@ # == Class: varnish # # Configure Varnish proxy cache # # === Parameters # # [*addrepo*] # Whether to add the official Varnish repos # [*secret*] # Varnish secret (used by varnishadm etc). # Optional; will be autogenerated if not specified # [*vcl_conf*] # Location of Varnish config file template # [*listen*] # IP address for HTTP to listen on # [*listen_port*] # Port to listen on for HTTP requests # [*admin_listen*] # IP address for admin requests - defaults to 127.0.0.1 # [*admin_port*] # Port for Varnish admin to listen on # [*min_threads*] # Varnish minimum thread pool size # [*max_threads*] # Varnish maximum thread pool size # [*thread_timeout*] # Thread timeout # [*storage_type*] # Whether to use malloc (RAM only) or file storage for cache # [*storage_size*] # Size of cache +# [*storage_additional*] +# Hash of additional storage backends containing strings in the varnish format (passed to -s) # [*varnish_version*] # Major Varnish version to use # [*vcl_reload*] # Script to use to load new Varnish config # [*package_ensure*] # Ensure specific package version for Varnish, eg 3.0.5-1.el6 # [*runtime_params*] # Hash of key:value runtime parameters # class varnish ( - $runtime_params = {}, - $addrepo = true, - $admin_listen = '127.0.0.1', - $admin_port = '6082', - $listen = '0.0.0.0', - $listen_port = '6081', - $secret = undef, - $secret_file = '/etc/varnish/secret', - $vcl_conf = '/etc/varnish/default.vcl', - $storage_type = 'file', - $storage_file = '/var/lib/varnish/varnish_storage.bin', - $storage_size = '1G', - $min_threads = '50', - $max_threads = '1000', - $thread_timeout = '120', - $varnish_version = '4.1', - $instance_name = undef, - $package_ensure = 'present', - $package_name = 'varnish', - $service_name = 'varnish', - $vcl_reload_cmd = undef, - $vcl_reload_path = $::path, + $runtime_params = {}, + $addrepo = true, + $admin_listen = '127.0.0.1', + $admin_port = '6082', + $listen = '0.0.0.0', + $listen_port = '6081', + $secret = undef, + $secret_file = '/etc/varnish/secret', + $vcl_conf = '/etc/varnish/default.vcl', + $storage_type = 'file', + $storage_file = '/var/lib/varnish/varnish_storage.bin', + $storage_size = '1G', + $storage_additional = [], + $min_threads = '50', + $max_threads = '1000', + $thread_timeout = '120', + $varnish_version = '4.1', + $instance_name = undef, + $package_ensure = 'present', + $package_name = 'varnish', + $service_name = 'varnish', + $vcl_reload_cmd = undef, + $vcl_reload_path = $::path, ) { if $package_ensure == 'present' { $version_major = regsubst($varnish_version, '^(\d+)\.(\d+).*$', '\1') $version_minor = regsubst($varnish_version, '^(\d+)\.(\d+).*$', '\2') $version_full = $varnish_version } else { $version_major = regsubst($package_ensure, '^(\d+)\.(\d+).*$', '\1') $version_minor = regsubst($package_ensure, '^(\d+)\.(\d+).*$', '\2') $version_full = "${version_major}.${version_minor}" if $varnish_version != "${version_major}.${version_minor}" { fail("Version mismatch, varnish_version is ${varnish_version}, but package_ensure is ${version_full}") } } include ::varnish::params if $vcl_reload_cmd == undef { $vcl_reload = $::varnish::params::vcl_reload } else { $vcl_reload = $vcl_reload_cmd } validate_bool($addrepo) validate_string($secret) validate_absolute_path($secret_file) unless is_integer($admin_port) { fail('admin_port invalid') } unless is_integer($min_threads) { fail('min_threads invalid') } unless is_integer($max_threads) { fail('max_threads invalid') } validate_absolute_path($storage_file) validate_hash($runtime_params) validate_re($storage_type, '^(malloc|file)$') validate_re("${version_major}.${version_minor}", '^[3-5]\.[0-9]') if $addrepo { class { '::varnish::repo': before => Class['::varnish::install'], } } include ::varnish::install class { '::varnish::secret': secret => $secret, require => Class['::varnish::install'], } class { '::varnish::config': require => Class['::varnish::secret'], notify => Class['::varnish::service'], } class { '::varnish::service': require => Class['::varnish::config'], } } diff --git a/templates/sysconfig.erb b/templates/sysconfig.erb index 9a504de..904ac9e 100644 --- a/templates/sysconfig.erb +++ b/templates/sysconfig.erb @@ -1,53 +1,56 @@ ############################################ # WARNING: # Managed by Puppet - DO NOT modify locally! ############################################ START=yes NFILES=131072 MEMLOCK=82000 RELOAD_VCL=1 VARNISH_VCL_CONF=<%= scope['::varnish::vcl_conf'] %> <% if scope['::varnish::version_major'] == 3 sep = ',' else sep = ' -a ' end if scope['::varnish::listen'].is_a?(Array) -%> VARNISH_LISTEN="-a <% scope['::varnish::listen'].each_with_index do |addr, idx| %><%= addr %><% unless addr.include?(':') %>:<%= scope['::varnish::listen_port'] %><% end -%><% if idx != (scope['::varnish::listen'].length-1) -%><%= sep %><% end -%><% end %>" <% else -%> VARNISH_LISTEN="-a <%= scope['::varnish::listen'] %>:<%= scope['::varnish::listen_port'] %>" <% end -%> VARNISH_ADMIN_LISTEN_ADDRESS=<%= scope['::varnish::admin_listen'] %> VARNISH_ADMIN_LISTEN_PORT=<%= scope['::varnish::admin_port'] %> VARNISH_SECRET_FILE=<%= scope['::varnish::secret_file'] %> VARNISH_MIN_THREADS=<%= scope['::varnish::min_threads'] %> VARNISH_MAX_THREADS=<%= scope['::varnish::max_threads'] %> VARNISH_THREAD_TIMEOUT=<%= scope['::varnish::thread_timeout'] %> VARNISH_STORAGE=<%= scope['::varnish::storage_type'] %>,<% if scope['::varnish::storage_type'] == 'file' -%><%= scope['::varnish::storage_file'] %>,<% end -%><%= scope['::varnish::storage_size'] %> VARNISH_TTL=120 DAEMON_OPTS="<% if scope['::varnish::params::service_provider'] == 'sysvinit' -%><%= scope['::varnish::config::jail_opt'] %> \<% end %> ${VARNISH_LISTEN} \ <% unless [nil, :undefined, :undef, ''].include?(scope['::varnish::instance_name']) -%> -n <%= scope['::varnish::instance_name'] %> \ <% end -%> -f ${VARNISH_VCL_CONF} \ -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \ -t ${VARNISH_TTL} \ <% if scope['::varnish::version_major'] == '3' -%> -w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \ <% else -%> -p thread_pool_min=${VARNISH_MIN_THREADS} \ -p thread_pool_max=${VARNISH_MAX_THREADS} \ -p thread_pool_timeout=${VARNISH_THREAD_TIMEOUT} \ <% end -%> <% scope['::varnish::runtime_params'].each do |k,v| -%> -p <%= k %>=<%= v %> \ <% end -%> -S ${VARNISH_SECRET_FILE} \ - -s ${VARNISH_STORAGE}" + -s ${VARNISH_STORAGE} \ +<% scope['::varnish::storage_additional'].each do |k| -%> + -s <%= k %> \ +<% end -%>" diff --git a/templates/varnish.service.erb b/templates/varnish.service.erb index 4faf9ef..3b979e3 100644 --- a/templates/varnish.service.erb +++ b/templates/varnish.service.erb @@ -1,47 +1,50 @@ [Unit] Description=Varnish HTTP accelerator [Service] Type=forking LimitNOFILE=131072 LimitMEMLOCK=82000 LimitCORE=infinity PrivateTmp=true ExecReload=<%= scope['::varnish::vcl_reload'] %> ExecStart=/usr/sbin/varnishd <%= scope['::varnish::config::jail_opt'] %> \ -P /var/run/varnish.pid \ -t 120 \ <% unless [nil, :undefined, :undef, ''].include?(scope['::varnish::instance_name']) -%> -n <%= scope['::varnish::instance_name'] %> \ <% end -%> -f <%= scope['::varnish::vcl_conf'] %> \ <% if scope['::varnish::version_major'] == 3 sep = ',' else sep = ' -a ' end if scope['::varnish::listen'].is_a?(Array) -%> -a <% scope['::varnish::listen'].each_with_index do |addr, idx| %><%= addr %><% unless addr.include?(':') %>:<%= scope['::varnish::listen_port'] %><% end -%><% if idx != (scope['::varnish::listen'].length-1) -%><%= sep %><% end -%><% end %> \ <% else -%> -a <%= scope['::varnish::listen'] %>:<%= scope['::varnish::listen_port'] %> \ <% end -%> -T <%= scope['::varnish::admin_listen'] %>:<%= scope['::varnish::admin_port'] %> \ <% if scope['::varnish::version_major'] == '3' -%> -w <%= scope['::varnish::min_threads'] %>,<%= scope['::varnish::max_threads'] %>,<%= scope['::varnish::thread_timeout'] %> \ <% else -%> -p thread_pool_min=<%= scope['::varnish::min_threads'] %> \ -p thread_pool_max=<%= scope['::varnish::max_threads'] %> \ -p thread_pool_timeout=<%= scope['::varnish::thread_timeout'] %> \ <% end -%> <% scope['::varnish::runtime_params'].each do |k,v| -%> -p <%= k %>=<%= v %> \ <% end -%> -S <%= scope['::varnish::secret_file'] %> \ - -s <%= scope['::varnish::storage_type'] %>,<% if scope['::varnish::storage_type'] == 'file' -%><%= scope['::varnish::storage_file'] %>,<% end -%><%= scope['::varnish::storage_size'] %> + -s <%= scope['::varnish::storage_type'] %>,<% if scope['::varnish::storage_type'] == 'file' -%><%= scope['::varnish::storage_file'] %>,<% end -%><%= scope['::varnish::storage_size'] %> \ +<% scope['::varnish::storage_additional'].each do |k| -%> + -s <%= k %> \ +<% end -%> [Install] WantedBy=multi-user.target