Setting up Kippo

kippo

#!/bin/bash
# /etc/init.d/kippo
 
### BEGIN INIT INFO
# Provides:   kippo
# Required-Start: $local_fs $remote_fs
# Required-Stop:  $local_fs $remote_fs
# Should-Start:   $network
# Should-Stop:    $network
# Default-Start:  2 3 4 5
# Default-Stop:   0 1 6
# Short-Description:    Kippo server
# Description:    Starts the kippo server
### END INIT INFO
 
# :author: paul schwendenman
#   :date: 02 oct 2013


# PATH TO TWISTED
# i.e Type "which twisd" in a shell with your virtualenv active
# No need to otherwise activate the virtualenv
PYTHON=
 
# Kippo project name (use if running more than one gunicorn application)
KIPPO_PROJECT="kippo"
 
# The pid file from the config file
PID_FILE=/path/to/pid_file.pid
 
# The kippo config file
KIPPO_CONF=/path/to/kippo.tac
 
# The kippo log file
PID_FILE=/path/to/kippo.log
 

# The kippo user
USERNAME=kippo
 

as_user() {
  if [ $(whoami) == $USERNAME ] ; then
    bash -c "$1"
  else
    su - $USERNAME -c "$1"
  fi
}
 
 
test_pid () {
	pgrep -F $1 &> /dev/null
}
 
ki_start() {
	if test_pid $PID_FILE
	then
		echo "$KIPPO_PROJECT was already running"
	else
		as_user $TWISTED -y $KIPPO_CONF -l $LOG_FILE --pidfile $PID_FILE
	fi
}
 
ki_kill() {
	if test_pid $PID_FILE
	then
		kill $(cat $PID_FILE)
		i=0
		while test_pid $PID_FILE
		do
			if [ $i = '0' ]
			then
				echo "... waiting"
			else
				echo -n "."
			fi
			i=$(($i + 1))
			sleep .1
		done
	else
		echo "$KIPPO_PROJECT was not running"
	fi
}
 
ki_reload() {
	if test_pid $PID_FILE
	then
		kill -HUP $(cat $PID_FILE)
	else
		echo "$KIPPO_PROJECT was not running"
	fi
 
}
 
case "$1" in
  start)
	echo -n "Starting $KIPPO_PROJECT... "
        ki_start
        ;;
  stop)
	echo -n "Stopping $KIPPO_PROJECT... "
        ki_kill
        ;;
  restart)
	echo -n "Restarting $KIPPO_PROJECT... "
        ki_kill
        ki_start
        ;;
  reload)
	echo -n "Reloading $KIPPO_PROJECT... "
        ki_reload
        ;;
  status)
	if test_pid $PID_FILE
	then
		echo "$KIPPO_PROJECT is running."
	else
		echo "$KIPPO_PROJECT is not running."
	fi
	;;
  *)
        echo "Usage: $0 {start|stop|restart|reload|status}" >&2
        exit 1
        ;;
esac

echo
exit 0

kippo.rst

1. Download kippo from tar ball on google code and run to see what type of hits you were getting.
2. Figure out that it works okay but most of the traffic is attempting sftp
3. Set up a fork of kipppo from github, that is really slow. figure that out with:

twistd -y kippo.tac -n --spew

4. Figure out the the process is hanging on the deep copy. Decide to try pypy
5. Install pypy
6. Check the speed of deepcopy

dc.py:

r = []
for i in xrange(40):
    l = [round(random.random()*100000) for i in xrange(10*3*60*60)]
    r.append(l)

ini = time.time()
copy.deepcopy(r)
print "Time: ", time.time() - ini

source:	https://bugs.pypy.org/issue767

7. Roughly follow these instructions

$ curl -O http://python-distribute.org/distribute_setup.py
$ curl -O https://raw.github.com/pypa/pip/master/contrib/get-pip.py
$ ./pypy-2.1/bin/pypy distribute_setup.py
$ ./pypy-2.1/bin/pypy get-pip.py
$ ./pypy-2.1/bin/pip install pygments  # for example

source:	http://pypy.readthedocs.org/en/latest/getting-started.html

8. Install 'twisted' and 'pycrypto'
9. Run kippo with:

$ pypy /usr/bin/twisted -y kippo.tac -n

10. Change the start.py file to use pypy.