Skip to content

Instantly share code, notes, and snippets.

@avnerbarr
Created July 3, 2013 12:39
Show Gist options
  • Save avnerbarr/5917546 to your computer and use it in GitHub Desktop.
Save avnerbarr/5917546 to your computer and use it in GitHub Desktop.
Installing StatsD to write to mongodb by a total systems noob - some what annoying thing to do and time time consuming because nothing seems to work the first time. There aren't that many configurations but you got to get them right. This gist is more of a walkthrough and less of an automated install like you would probably wish there was. There…
I installed Statsd on Ubuntu 12.04.2 LTS (GNU/Linux 3.2.0-48-virtual x86_64)
Get node (i think that the repo for it changed so check out a different way to install)
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python-software-properties git-core
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs
Clone the git for statsd (If doesn't work install git :))
cd /tmp
git clone https://github.com/etsy/statsd.git
that will clone to /tmp/statsd
for some reason I saw that lots of docs assume that statsd is in /opt/statsd... so lets copy it there:
sudo mkdir /opt/statsd
then just copy all the files from /tmp/statsd to /opt/statsd...
sudo cp -r /tmp/statsd/* /opt/statsd
(or you could have just cloned there right away... but you might skip that step ... in any case could have done
cd /opt
sudo git clone git://github.com/etsy/statsd.git
)
now need to configure statsd to write to mongo:
Do you have mongo installed?
If not find some instructions - its pretty easy but follow them closely:
After mongo is up and running you need to install the mongodb backend module for statsd:
follow these instructions https://github.com/dynmeth/mongo-statsd-backend
Or cross your fingers and try this:
cd /opt/statsd
(sudo?) npm install mongo-statsd-backend
Don't know how, but npm install mongo-statsd-backend knew that I copied statsd to /opt/statsd (or maybe that is the only place that it will look to copy it too - hence the pain in the ass I talked about in the title of this gist - took me a bunch of times to get this part to work!!! (might not have done cd /opt/statsd the first 10 times...)
after that we need to configure StatsD to use this plugin:
do:
sudo cp /tmp/localConfig.js /opt/statsd/localConfig.js
(I found file names could change, so worst case just create some file, for instance:
sudo touch /opt/statsd/my_config_that_uses_mongodb.js)
then edit it as so:
sudo vi /opt/statsd/myconfig.js (or localConfig.js - what ever you decided to call it)
Copy this into your config file
(set the stuff to your mongo host and ports etc. from the first step)
In the backends array you got to look around where the module was installed (mine is in /opt/statsd/node_modules/mongo-statsd-backend/lib/index.js
Make sure it is in the square brackets and in between '' marks - another hint: notice that node_modules directory)
{
mongoHost: 'user:pass@localhost',
mongoPort: 27017,
mongoMax: 2160,
mongoPrefix: true,
mongoName: 'databaseName',
backends: ['/path/to/module/lib/index.js']
}
Now we can either run as a user process (easy for testing but if the machine crashes - you log out , whatever bad can happen the server will stop...)
by doing
node stats.js /path/to/config
Some Tips:
1) Make sure that the ports are open on your machine for passing data to statsd (If you plan on doing it from another machine of course)
2) If you want a particular port (not the default 8125) than put in the config file " port: 8125, "(which ever port you want) and make sure your firewall is open on UDP to that port
Testing:
Thank god, we can now put our dev cap back on can forget about all this system boredom.
Find an appropriate client for statsd in you development language and start sending it some data. Check that your mongo instance is getting the data!
Better:
You probably want to run statsd as a service (monitored by the OS)
So we need to create a init file (or what ever you use in your environment)
I did this using a tool called upstart (total noob I admit, don't know what these tools are) and this tutorial
http://kvz.io/blog/2009/12/15/run-nodejs-as-a-service-on-ubuntu-karmic/
Basic idea is as follows:
You create a script called statsd.conf (or what ever name you desire instead of "statsd")
You put this script in /etc/init (True for ununto OS- I believe other nx have different locations)
and then start/stop them using "start statsd" and "stop statsd"
in that script put instructions like this:
#Script start - Don't put this line in the file you created -
#!upstart
description "Statsd node.js server"
author "Your name"
start on started mountall
stop on shutdown
# Automatically Respawn:
respawn
respawn limit 99 5
script
export HOME="/root"
echo $$ > /var/run/statsd.pid
exec sudo /usr/bin/node /opt/statsd/stats.js /opt/statsd/localConfig.js >> /var/log/statsd.log 2> /var/log/statsd.error.log
end script
pre-start script
# Date format same as (new Date()).toISOString() for consistency
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Starting" >> /var/log/statsd.log
end script
pre-stop script
rm /var/run/statsd.pid
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Stopping" >> /var/log/statsd.log
######################################
### End of script
#######################################
IMPORTANT! Notice this line which is essentially what the system will run:
exec sudo /usr/bin/node /opt/statsd/stats.js /opt/statsd/localConfig.js >> /var/log/statsd.log 2> /var/log/statsd.error.log
Here you will see that node is hard coded - so you might need to change that per your system (do ' which node' or 'which nodejs' etc.)
Notice where /opt/statsd/stats.js is located on your machine
notice where /opt/statsd/localConfig.js is located on your machine
and fix accordingly
Notice where stdout is flushed to (/var/log/statsd.log) and the error log (/var/log/statsd.error)
So you can check out that all is running.
Thats it - First time took me 5 hours - second time 3 minutes... Hope this saves you some time
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment