Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
#!/bin/bash -e
# Neil Williams, reddit
# configuration that doesn't change much
APTITUDE_OPTIONS="-y" # limit bandwidth: -o Acquire::http::Dl-Limit=100"
# don't blunder on if an error occurs
set -e
# validate some assumptions of the script
if [ $(id -u) -ne 0 ]; then
echo "ERROR: Must be run with root privileges."
exit 1
source /etc/lsb-release
if [ "$DISTRIB_ID" != "Ubuntu" ]; then
echo "ERROR: Unknown distribution $DISTRIB_ID. Only Ubuntu is supported."
exit 1
if [ "$DISTRIB_RELEASE" != "11.04" ]; then
echo "ERROR: Only Ubuntu 11.04 is supported."
exit 1
# get configured
echo "Welcome to the reddit install script!"
read -ei $REDDIT_USER -p "What user should reddit run as? " REDDIT_USER
read -ei $REDDIT_HOME -p "What directory should it be installed in? " REDDIT_HOME
echo "Beginning installation. This may take a while..."
# create the user if non-existent
if ! id $REDDIT_USER > /dev/null; then
adduser --system $REDDIT_USER
# add some external ppas for packages
DEBIAN_FRONTEND=noninteractive aptitude install $APTITUDE_OPTIONS python-software-properties
apt-add-repository ppa:jason-alioth/reddit
# pin the ppa
cat <<HERE > /etc/apt/preferences.d/reddit
Package: *
Pin: release o=LP-PPA-jason-alioth-reddit
Pin-Priority: 600
# grab the new ppas' package listings
aptitude update
# install prerequisites
DEBIAN_FRONTEND=noninteractive aptitude install $APTITUDE_OPTIONS python-dev python-setuptools python-imaging python-pycaptcha python-mako python-nose python-decorator python-formencode python-pastescript python-beaker python-webhelpers python-amqplib python-pylibmc python-pycountry python-psycopg2 python-cssutils python-beautifulsoup python-sqlalchemy cython python-pybabel python-tz python-boto python-lxml python-pylons python-pycassa python-recaptcha gettext make optipng uwsgi uwsgi-core uwsgi-plugin-python nginx git-core python-profiler memcached postgresql postgresql-client curl daemontools daemontools-run rabbitmq-server cassandra
# grab the source
if [ ! -d $REDDIT_HOME ]; then
if [ ! -d $REDDIT_HOME/reddit ]; then
sudo -u $REDDIT_USER git clone $REDDIT_REPO
if [ ! -d $REDDIT_HOME/reddit-i18n ]; then
sudo -u $REDDIT_USER git clone $I18N_REPO
# wait a bit to make sure all the servers come up
sleep 30
# configure cassandra
if ! echo | cassandra-cli -h localhost -k reddit > /dev/null 2>&1; then
echo "create keyspace reddit;" | cassandra-cli -h localhost -B
echo "create column family permacache with column_type = 'Standard' and comparator = 'BytesType';" | cassandra-cli -B -h localhost -k reddit || true
# set up postgres
IS_DATABASE_CREATED=$(sudo -u postgres psql -t -c "SELECT COUNT(1) FROM pg_catalog.pg_database WHERE datname = 'reddit';")
if [ $IS_DATABASE_CREATED -ne 1 ]; then
cat <<PGSCRIPT | sudo -u postgres psql
CREATE USER reddit WITH PASSWORD 'password';
sudo -u postgres psql reddit < $REDDIT_HOME/reddit/sql/functions.sql
# set up rabbitmq
if ! rabbitmqctl list_vhosts | egrep "^/$"
rabbitmqctl add_vhost /
if ! rabbitmqctl list_users | egrep "^reddit"
rabbitmqctl add_user reddit reddit
rabbitmqctl set_permissions -p / reddit ".*" ".*" ".*"
# run the reddit setup script
cd $REDDIT_HOME/reddit/r2
sudo -u $REDDIT_USER make pyx # generate the .c files from .pyx
sudo -u $REDDIT_USER python build
python develop
# run the i18n setup script
cd $REDDIT_HOME/reddit-i18n/
sudo -u $REDDIT_USER python build
python develop
sudo -u $REDDIT_USER make
# do the r2 build after languages are installed
cd $REDDIT_HOME/reddit/r2
sudo -u $REDDIT_USER make
# install the daemontools runscripts
cd $REDDIT_HOME/reddit/r2
if [ ! -L run.ini ]; then
sudo -u $REDDIT_USER ln -s example.ini run.ini
ln -s $REDDIT_HOME/reddit/srv/* /etc/service/ || true
/sbin/start svscan || true
# set up uwsgi
cat >/etc/uwsgi/apps-available/reddit.ini <<UWSGI
; master / uwsgi protocol configuration
plugins = python
master = true
master-as-root = true
vacuum = true
limit-post = 512000
buffer-size = 8096
chdir = $REDDIT_HOME/reddit/r2
socket = /tmp/reddit.sock
chmod-socket = 666
; worker configuration
lazy = true
max-requests = 10000
processes = 1
; app configuration
paste = config:$REDDIT_HOME/reddit/r2/run.ini
if [ ! -L /etc/uwsgi/apps-enabled/reddit.ini ]; then
ln -s /etc/uwsgi/apps-available/reddit.ini /etc/uwsgi/apps-enabled/reddit.ini
/etc/init.d/uwsgi start
# set up nginx
cat >/etc/nginx/sites-available/reddit <<NGINX
include uwsgi_params;
uwsgi_param SCRIPT_NAME "";
server {
listen 8000;
location / {
uwsgi_pass unix:///tmp/reddit.sock;
gzip on;
gzip_vary on;
gzip_proxied any;
gzip_min_length 100;
gzip_comp_level 4;
gzip_types text/plain text/css text/javascript text/xml application/json text/csv application/x-javascript application/xml application/xml+rss;
if [ ! -L /etc/nginx/sites-enabled/reddit ]; then
ln -s /etc/nginx/sites-available/reddit /etc/nginx/sites-enabled/reddit
/etc/init.d/nginx restart
# install the crontab
crontab -u $REDDIT_USER -l > $CRONTAB || true
# m h dom mon dow command
*/5 * * * * $REDDIT_HOME/reddit/scripts/
*/4 * * * * $REDDIT_HOME/reddit/scripts/
*/3 * * * * $REDDIT_HOME/reddit/scripts/
1 * * * * $REDDIT_HOME/reddit/scripts/
0 23 * * * $REDDIT_HOME/reddit/scripts/
30 23 * * * $REDDIT_HOME/reddit/scripts/
# done!
echo "Done installing reddit!"

dweitz commented Jun 14, 2011

The link to the reddit version of Paste (line 132) doesn't work anymore. Any other place to download it?


spladug commented Jun 14, 2011

The package hangs out here. That should've automatically been sourced by our Did that not work?

dweitz commented Jun 15, 2011

It was timing out for me before for that one file and giving me 404 errors when I tried to download from a browser. Seems to be working now.

Ran into this error when I tried to install on a fresh ubuntu:
paster run standalone r2/lib/ -c "build_reddit_js()"
Traceback (most recent call last):
File "/usr/local/bin/paster", line 9, in
load_entry_point('PasteScript==1.7.3', 'console_scripts', 'paster')()
File "/usr/local/lib/python2.6/dist-packages/PasteScript-1.7.3-py2.6.egg/paste/script/", line 83, in run
command = commands[command_name].load()
File "/usr/lib/python2.6/dist-packages/", line 1948, in load
entry = import(self.module_name, globals(),globals(), ['name'])
File "/home/reddit/reddit/r2/r2/", line 26, in
from r2.config.middleware import make_app
File "/home/reddit/reddit/r2/r2/config/", line 22, in
from r2.lib.cache import sgm
File "/home/reddit/reddit/r2/r2/lib/", line 35, in
from r2.lib.utils import in_chunks, prefix_keys, trace
File "/home/reddit/reddit/r2/r2/lib/utils/", line 22, in
from utils import *
File "/home/reddit/reddit/r2/r2/lib/utils/", line 36, in
from r2.lib.filters import _force_unicode
File "/home/reddit/reddit/r2/r2/lib/", line 29, in
from lxml.sax import saxify
ImportError: No module named lxml.sax
make: *** [r2/public/static/button.js] Error 1

had to manually install lxml for it to work, with
sudo apt-get install python-lxml

The newest version of pycountry 0.14 which is installed by default doesn't seem to work with this script, threw this error
File '/usr/local/lib/python2.6/dist-packages/pycountry-0.14-py2.6.egg/pycountry/', line 16 in
class Countries(pycountry.db.Database):
AttributeError: 'module' object has no attribute 'db'

installing 0.12.1 fixed it
easy_install "pycountry==0.12.1"

For this to still work in (at least Ubuntu 10.10), you should change Cassandra's package name to include a previous version, since getting Reddit up in version 0.8 with the current configs wont work. "cassandra=0.7.2-0ubuntu4" would work.

pycountry has been updated and now works with the latest version

rpicard commented Sep 28, 2011

I ran into some problems using this on Ubuntu 11.04. I could be doing something wrong but I kept getting an error saying that aptitude was not a recognized command. I changed all of the references from aptitude to apt-get and it seems to have solved the problem. I just installed Ubuntu today to start playing with Reddit. I think the problem is that it doesn't come with aptitude installed (and it did in previous versions?).

See this gist for the modified version.

aptitude isn't installed by default in 11.04. I BELIEVE it's something of an upgrade to apt-get, but I can't fully confirm that.

rpicard commented Sep 28, 2011

I figured it was something like that. I ran into a few more errors after I got passed that and I realized I was getting in a little over my head with this. I'm going to stick with simple PHP scripts for now.

Munksey commented Oct 11, 2011

Aptitude is not installed by default. I just did a fresh install of 11.04. But, you can easily install it by running:
apt-get update
apt-get install aptitude

I'm attempting to install on 11.1 (I removed the bit in the .sh that checks for version 11.04) and the install bails when it gets to the PPA:

You are about to add the following PPA to your system:
Packages for reddit
More info:

(everything is going fine until... )

Hit oneiric-backports/multiverse TranslationIndex
Hit oneiric-backports/restricted TranslationIndex
Err oneiric/main Sources
404 Not Found
Err oneiric/main i386 Packages
404 Not Found
Hit oneiric-backports/universe TranslationIndex
Hit oneiric/main Translation-en
Hit oneiric/multiverse Translation-en
Hit oneiric/restricted Translation-en
Hit oneiric/universe Translation-en
Hit oneiric-updates/main Translation-en
Hit oneiric-updates/multiverse Translation-en
Hit oneiric-updates/restricted Translation-en
Hit oneiric-updates/universe Translation-en
Hit oneiric-backports/main Translation-en
Hit oneiric-backports/multiverse Translation-en
Hit oneiric-backports/restricted Translation-en
Hit oneiric-backports/universe Translation-en
Ign oneiric/main Translation-en_US
Ign oneiric/main Translation-en


Any ideas on what's causing the script to bail?


spladug commented Nov 8, 2011

Yup, none of the packages in the PPA are compiled for Oneiric. There's a reason that version check is there :P

Silly Me.

After reinstalling the correct version of Ubuntu, I'm bailing on the same error as Cheapsteak:

ImportError: No module named lxml.sax
make: *** [r2/public/static/button.js] Error 1

I've manually installed lxml with "sudo apt-get install python-lxml" but the script still dies at this line.

ttanner commented Dec 1, 2011

I've tried to follow the steps on Lucid (10.04.3 LTS) but it fails on
$ make pyx
Traceback (most recent call last):
File "r2/lib/", line 8, in
from r2.lib.i18n import get_available_languages
ImportError: No module named r2.lib.i18n
Traceback (most recent call last):
File "r2/lib/", line 8, in
from r2.lib.i18n import get_available_languages
ImportError: No module named r2.lib.i18n
cython r2/models/_builder.pyx

Error converting Pyrex file to C:

def _tree_filter(self, x):
return tree_sort_fn(x) < self.after._id

def _viewable_message(self, m):
    if (c.user_is_admin
        or getattr(m, "author_id", 0) == c.user._id

/home/reddit/reddit/r2/r2/models/_builder.pyx:282:22: Call with wrong number of arguments (expected 2, got 3)


spladug commented Dec 1, 2011

You need a newer version of Cython.

Note: we don't support Lucid any more and there're parts of the code that will not work on Python 2.6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment