Hack Reactor Talk

hack-reactor.md

Tales From a Heroku User

Here are some things I have learned along the way.

Last Updated: 2013-02-08

Original Audience: Hack Reactor

About

• @ryandotsmith
• Engineer at Heroku for almost 3 years.
• Programming since university.
• Studied Math.
• Built armature radios in junior high.

Things I have worked on

• The Heroku API
• Message queues (e.g. queue_classic)
• Heroku's usage & billing system
• DNS at Heroku
• Log delivery and visibility tooling (e.g. l2met, log-shuttle)

Technologies that I spend considerable time with

• Heroku
• PostgreSQL
• Redis
• Ruby
• Go
• UNIX

Heroku

Things Heroku makes trivial:

• Keeping a process alive.
• Keeping your data alive.
• Routing HTTP requests to more than one process.
• Adding new processes and new apps.
• SSL Termination.
• Operating System Maintenance.
• Log routing.
• Security.

At one time in our history, everyone made their own electricity.

A dynamo that would power a business. The business would employ people to build and maintain the dynamo. Dyanmos were abandoned as they were neither safe or economical. I feel that this is similar to servers and Heroku.

How Heroku works

This is a great time for questions. Stop me at any point and we can take a deep dive into a particular concept or system.

How to design your app knowing how Heroku works

Stateless & Uncoordinated

• Processes are distributed
• Your database might be distributed
• l2met example

Web Processes are Precious

• Handle work in the background
• worker pattern example

12 Factor

12factor.net

• Good for any app
• Forced if running on Heroku

Workflow

I find that Heroku is most powerful when you learn to embrace the UNIX toolkit.

Config

There are no configuration files. Files are clunky and hard to manage, the unix way of dealing with config is to use the UNIX environment memory.

$ man env

Grabbing the config from your Heroku app.

$ heroku config -s > .env

Loading your config locally.

$ export $(cat .env)

Copying config from one app to another.

$ heroku config:set $(heroku config -s)

Log Analysis

Don't think of your logs as static files. Think of the logs as live streams of your app's health and status.

$ ruby -e '$stdout.puts("at=\"hack-reactor\"")'

Counting the number of HTTP requests per second.

$ heroku logs -t -p router | pv -lr >/dev/null

Sampling your response times.

$ heroku logs -t -p router | awk '{print $10,$11,$12}'

Using tools like log-shuttle and l2met we can build charts of system metrics.

Interacting with your database

A database provided by the Heroku addons catalog is just a database. With Heroku Postgresql, you can use standard UNIX tools to interact with it.

Connecting to your database.

$ export url=`heroku config -s | grep "DATABASE_URL" | sed 's/DATABASE_URL=//'`
$ psql $url

Using your editor to build queries.

$ export EDITOR=vi
$ psql $url
database - \e

Similar feature available in bash.

$ echo 'hello world' | grep 'hello' | sed 's/hello/bye'
$ fc

Writing query results to your local filesystem.

$ echo 'select 1 \g out.txt' | psql $url
$ cat out.txt

Getting Real

It is clear that Heroku is the best place to develop your application, but it is also a great place to run and scale your application. A couple of things to keep in mind when you grow your traffic and your uptime.

• Choose the right ratio of work between your process and your database.
• Loading functions in your database is OK. Be aware that it won't scale horizontally.
• Scaling horizontall can help increase throughput when your latencies are high.
• Think about latency vs. throughput and how to balance the two.

Maximizing the uptime of your Heroku app

Scaling a database's reads is pretty easy - just rack on a follower or three. It's scaling writes that's difficult. On the other hand, if you're doing more writes than a Mecha can handle you're in pretty rare company.