Skip to content

Instantly share code, notes, and snippets.

@holyjak
Last active August 29, 2015 14:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save holyjak/476b2d3c79f00a465304 to your computer and use it in GitHub Desktop.
Save holyjak/476b2d3c79f00a465304 to your computer and use it in GitHub Desktop.
Configure the HTTP traffic replicator [gor](https://github.com/buger/gor) to run on an AWS Elastic Beanstalk instance as a daemon. Store this file as `./ebextensions/10gor.config`
# File: .ebextensions/10gor.config
# Config Gor to copy a sample of Prod http traffice to staging
files:
# Utility for daemonizing binaries such as gor; see http://libslack.org/daemon
/opt/daemon.rpm:
source: "https://s3-eu-west-1.amazonaws.com/elasticbeanstalk-eu-west-1-<our id>/our_fileserver/daemon-0.6.4-1.x86_64.rpm"
authentication: S3Access # See AWS::CloudFormation::Authentication below
owner: root
group: root
# daemon config so that we don't need to repeat these command line options and
# can just use the service's name ("gor")
# We need to intercept the port 8080, not 80 (that iptables redirect to 8080)
/etc/daemon.conf:
# Troubleshooting tips:
# 1) Send stderr/out output of both daemon and the service to a file - see the commented-out line
# (it should be also possible to send it to syslog but that did not work for me)
# 2) Add "foreground" to the options and start the service manually on the server
content: |
gor respawn,command=/opt/gor --input-raw :8080 --output-http 'https://our-staging.example.com|1'
#gor output=/var/log/gor.log,respawn,command=/opt/gor --stats --output-http-stats --input-raw :8080 --output-http 'https://our-staging.example.com|1'
#gor errlog=/var/log/daemonapp.log,dbglog=/var/log/daemonapp.log,output=/var/log/gor.log,verbose,debug,respawn,command=/opt/gor --verbose --stats --output-http-stats --input-raw :8080 --output-http 'https://our-staging.example.com|1'
# Use the commented-out line to enable stats logging to syslog (/var/log/messages) every 5s for troubleshooting
content: |
gor respawn,command=/opt/gor --input-raw :8080 --output-http 'https://our-staging.example.com|1'
#gor respawn,output=gor.info,command=/opt/gor --stats --output-http-stats --input-raw :8080 --output-http 'https://our-staging.example.com|1'
# HTTP traffic replicator; see https://github.com/buger/gor
/opt/gor:
source: "https://s3-eu-west-1.amazonaws.com/elasticbeanstalk-eu-west-1-<our id>/our_fileserver/gor"
authentication: S3Access
mode: "000755"
owner: root
group: root
# System V service that supports chkconfig
/etc/init.d/gor:
mode: "000755"
owner: root
group: root
content: |
## The chkconfig <levels> <startup order> <stop order> + descr. needed to
## support ensureRunning
# chkconfig: 345 92 08
# description: Gor copies traffic to staging
### BEGIN INIT INFO
# Provides: gor
# Short-Description: Start Gor to copy traffic to staging
### END INIT INFO
# See how we were called.
case "$1" in
start)
/usr/local/bin/daemon --name gor
;;
stop)
/usr/local/bin/daemon --name gor --stop
;;
status)
if ! /usr/local/bin/daemon --name gor --running; then
echo "gor is stopped"; exit 3
fi
;;
restart)
/usr/local/bin/daemon --name gor --stop
/usr/local/bin/daemon --name gor
;;
*)
echo $"Usage: $0 {start|stop|restart}"
exit 2
esac
exit 0
commands:
"Install daemon":
command: rpm -i /opt/daemon.rpm
test: /bin/sh -c "! rpm -q daemon" # It seems 'test: ! rpm -q daemon' ignores the !
services:
sysvinit:
gor:
enabled: true
ensureRunning: true
# Allow files: to access our S3 bucket
# See https://forums.aws.amazon.com/thread.jspa?messageID=557993
# BEWARE: We have to explicitely allow access to any subdirectory by
# editing the S3 bucket's policy and adding the subdir to the allowed Resources
AWSEBAutoScalingGroup:
Metadata:
AWS::CloudFormation::Authentication:
"S3Access": # reference this in the "authentication" property
type: S3
roleName: aws-elasticbeanstalk-ec2-role
buckets: elasticbeanstalk-eu-west-1-<our id>
@holyjak
Copy link
Author

holyjak commented Jul 30, 2015

Highlights

  1. Use the daemon utility to run Gor as a daemon (which it does not support out of the box). Daemon is small and works well. It will ignore gor's output and automatically restart it if it dies.
  2. Create an init.d script for gor. To support ebextensions's ensureRunning, it has to support chkconfig
  3. The test for whether daemon is installed cannot be just ! rpm -q daemon but needs to be /bin/sh -c "! rpm -q daemon"; the test property seems to require a single command to execute
  4. The files are downloaded from a private S3 bucket (which needs to be accessible by the EC2 role used and have the policy to allow access to the files in question)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment