diff --git a/README.md b/README.md index e73ce6d..2a0267c 100644 --- a/README.md +++ b/README.md @@ -1,62 +1,55 @@ # Sentry 10 On-Premise BETA [![Build Status][build-status-image]][build-status-url] Official bootstrap for running your own [Sentry](https://sentry.io/) with [Docker](https://www.docker.com/). **NOTE:** If you are not installing Sentry from scratch, our recommendation is to visit [On-Premise Stable for Sentry 9.1.2](https://github.com/getsentry/onpremise/tree/stable) as this version may not be fully backward compatible. If you still want to try it out make sure you are on 9.1.2 first, back up your old Docker volumes just in case, and remember that if you haven't set up Redis persistency yourself some of your data (like your stats) may be lost during the upgrade. ## Requirements * Docker 17.05.0+ * Compose 1.19.0+ ## Minimum Hardware Requirements: * You need at least 2400MB RAM ## Setup To get started with all the defaults, simply clone the repo and run `./install.sh` in your local check-out. There may need to be modifications to the included example config files (`sentry/config.example.yml` and `sentry/sentry.conf.example.py`) to accommodate your needs or your environment (such as adding GitHub credentials). If you want to perform these, do them before you run the install script and copy them without the `.example` extensions in the name (such as `sentry/sentry.conf.py`) before running the `install.sh` script. The recommended way to customize your configuration is using the files below, in that order: * `config.yml` * `sentry.conf.py` * `.env` w/ environment variables We currently support a very minimal set of environment variables to promote other means of configuration. If you have any issues or questions, our [Community Forum](https://forum.sentry.io/c/on-premise) is at your service! ## Event Retention Sentry comes with a cleanup cron job that prunes events older than `90 days` by default. If you want to change that, you can change the `SENTRY_EVENT_RETENTION_DAYS` environment variable in `.env` or simply override it in your environment. If you do not want the cleanup cron, you can remove the `sentry-cleanup` service from the `docker-compose.yml`file. ## Securing Sentry with SSL/TLS If you'd like to protect your Sentry install with SSL/TLS, there are fantastic SSL/TLS proxies like [HAProxy](http://www.haproxy.org/) and [Nginx](http://nginx.org/). You'll likely want to add this service to your `docker-compose.yml` file. ## Updating Sentry -Updating Sentry using Compose is relatively simple. Just use the following steps to update. Make sure that you have the latest version set in your Dockerfile. Or use the latest version of this repository. - -Use the following steps after updating this repository or your Dockerfile: -```sh -docker-compose build --pull # Build the services again after updating, and make sure we're up to date on patch version -docker-compose run --rm web upgrade # Run new migrations -docker-compose up -d # Recreate the services -``` +The included `install.sh` script is meant to be idempotent and to bring you to the latest version. What this means is you can and should run `install.sh` to upgrade to the latest version available. ## Resources * [Documentation](https://docs.sentry.io/server/installation/docker/) * [Bug Tracker](https://github.com/getsentry/onpremise/issues) * [Forums](https://forum.sentry.io/c/on-premise) * [IRC](irc://chat.freenode.net/sentry) (chat.freenode.net, #sentry) [build-status-image]: https://api.travis-ci.com/getsentry/onpremise.svg?branch=master [build-status-url]: https://travis-ci.com/getsentry/onpremise diff --git a/install.sh b/install.sh index 39349bc..742d131 100755 --- a/install.sh +++ b/install.sh @@ -1,157 +1,161 @@ #!/usr/bin/env bash set -e MIN_DOCKER_VERSION='17.05.0' MIN_COMPOSE_VERSION='1.19.0' MIN_RAM=2400 # MB SENTRY_CONFIG_PY='sentry/sentry.conf.py' SENTRY_CONFIG_YML='sentry/config.yml' SENTRY_EXTRA_REQUIREMENTS='sentry/requirements.txt' DID_CLEAN_UP=0 # the cleanup function will be the exit point cleanup () { if [ "$DID_CLEAN_UP" -eq 1 ]; then return 0; fi echo "Cleaning up..." docker-compose stop &> /dev/null DID_CLEAN_UP=1 } trap cleanup ERR INT TERM echo "Checking minimum requirements..." DOCKER_VERSION=$(docker version --format '{{.Server.Version}}') COMPOSE_VERSION=$(docker-compose --version | sed 's/docker-compose version \(.\{1,\}\),.*/\1/') RAM_AVAILABLE_IN_DOCKER=$(docker run --rm busybox free -m 2>/dev/null | awk '/Mem/ {print $2}'); # Compare dot-separated strings - function below is inspired by https://stackoverflow.com/a/37939589/808368 function ver () { echo "$@" | awk -F. '{ printf("%d%03d%03d", $1,$2,$3); }'; } # Thanks to https://stackoverflow.com/a/25123013/90297 for the quick `sed` pattern function ensure_file_from_example { if [ -f "$1" ]; then echo "$1 already exists, skipped creation." else echo "Creating $1..." cp -n $(echo "$1" | sed 's/\.[^.]*$/.example&/') "$1" fi } if [ $(ver $DOCKER_VERSION) -lt $(ver $MIN_DOCKER_VERSION) ]; then echo "FAIL: Expected minimum Docker version to be $MIN_DOCKER_VERSION but found $DOCKER_VERSION" exit -1 fi if [ $(ver $COMPOSE_VERSION) -lt $(ver $MIN_COMPOSE_VERSION) ]; then echo "FAIL: Expected minimum docker-compose version to be $MIN_COMPOSE_VERSION but found $COMPOSE_VERSION" exit -1 fi if [ "$RAM_AVAILABLE_IN_DOCKER" -lt "$MIN_RAM" ]; then echo "FAIL: Expected minimum RAM available to Docker to be $MIN_RAM MB but found $RAM_AVAILABLE_IN_DOCKER MB" exit -1 fi -# Ensure nothing is working while we install/update -docker-compose stop +# Clean up old stuff and ensure nothing is working while we install/update +docker-compose down --rmi local --remove-orphans echo "" echo "Creating volumes for persistent storage..." echo "Created $(docker volume create --name=sentry-data)." echo "Created $(docker volume create --name=sentry-postgres)." echo "Created $(docker volume create --name=sentry-redis)." echo "Created $(docker volume create --name=sentry-zookeeper)." echo "Created $(docker volume create --name=sentry-kafka)." echo "Created $(docker volume create --name=sentry-clickhouse)." echo "Created $(docker volume create --name=sentry-symbolicator)." echo "" ensure_file_from_example $SENTRY_CONFIG_PY ensure_file_from_example $SENTRY_CONFIG_YML ensure_file_from_example $SENTRY_EXTRA_REQUIREMENTS echo "" echo "Generating secret key..." # This is to escape the secret key to be used in sed below SECRET_KEY=$(head /dev/urandom | tr -dc "a-z0-9@#%^&*(-_=+)" | head -c 50 | sed -e 's/[\/&]/\\&/g') sed -i -e 's/^system.secret-key:.*$/system.secret-key: '"'$SECRET_KEY'"'/' $SENTRY_CONFIG_YML echo "Secret key written to $SENTRY_CONFIG_YML" echo "" echo "Building and tagging Docker images..." echo "" # Build the sentry onpremise image first as it is needed for the cron image +docker-compose pull --ignore-pull-failures +docker pull ${SENTRY_IMAGE:-getsentry/sentry:latest} docker-compose build --force-rm web docker-compose build --force-rm echo "" echo "Docker images built." # Very naively check whether there's an existing sentry-postgres volume and the PG version in it if [[ $(docker volume ls -q --filter name=sentry-postgres) && $(docker run --rm -v sentry-postgres:/db busybox cat /db/PG_VERSION 2>/dev/null) == "9.5" ]]; then + docker volume rm sentry-postgres-new || true # If this is Postgres 9.5 data, start upgrading it to 9.6 in a new volume docker run --rm \ -v sentry-postgres:/var/lib/postgresql/9.5/data \ -v sentry-postgres-new:/var/lib/postgresql/9.6/data \ tianon/postgres-upgrade:9.5-to-9.6 # Get rid of the old volume as we'll rename the new one to that docker volume rm sentry-postgres docker volume create --name sentry-postgres # There's no rename volume in Docker so copy the contents from old to new name # Also append the `host all all all trust` line as `tianon/postgres-upgrade:9.5-to-9.6` # doesn't do that automatically. docker run --rm -v sentry-postgres-new:/from -v sentry-postgres:/to alpine ash -c \ "cd /from ; cp -av . /to ; echo 'host all all all trust' >> /to/pg_hba.conf" # Finally, remove the new old volume as we are all in sentry-postgres now docker volume rm sentry-postgres-new fi echo "" echo "Setting up database..." if [ $CI ]; then docker-compose run --rm web upgrade --noinput echo "" echo "Did not prompt for user creation due to non-interactive shell." echo "Run the following command to create one yourself (recommended):" echo "" echo " docker-compose run --rm web createuser" echo "" else docker-compose run --rm web upgrade fi -SENTRY_DATA_NEEDS_MIGRATION=$(docker run --rm -v sentry-data:/data alpine ash -c "[ ! -d '/data/files' ] && ls -A1x /data | wc -l") +SENTRY_DATA_NEEDS_MIGRATION=$(docker run --rm -v sentry-data:/data alpine ash -c "[ ! -d '/data/files' ] && ls -A1x /data | wc -l || true") if [ "$SENTRY_DATA_NEEDS_MIGRATION" ]; then echo "Migrating file storage..." - docker run --rm -v sentry-data:/data alpine ash -c \ + # Use the web (Sentry) image so the file owners are kept as sentry:sentry + docker-compose run --rm --entrypoint /bin/bash web -c \ "mkdir -p /tmp/files; mv /data/* /tmp/files/; mv /tmp/files /data/files" fi echo "Boostrapping Snuba..." docker-compose up -d kafka redis clickhouse until $(docker-compose run --rm clickhouse clickhouse-client -h clickhouse --query="SHOW TABLES;" | grep -q sentry_local); do # `bootstrap` is for fresh installs, and `migrate` is for existing installs # Running them both for both cases is harmless so we blindly run them docker-compose run --rm snuba-api bootstrap --force || true; docker-compose run --rm snuba-api migrate || true; done; echo "" set -o allexport source .env set +o allexport echo "Migrating old events for the last $SENTRY_EVENT_RETENTION_DAYS days..." docker-compose run --rm web django backfill_eventstream --no-input --last-days $SENTRY_EVENT_RETENTION_DAYS echo "" cleanup echo "" echo "----------------" echo "You're all done! Run the following command to get Sentry running:" echo "" echo " docker-compose up -d" echo ""