crowdmatt

==  RemoveNameFromOrganizations: migrating ====================================
-- remove_column(:organizations, :name)
   -> 0.1580s
-- add_index(:organizations, :domain, {:unique=>true})
rake aborted!
An error has occurred, all later migrations canceled:

ActiveRecord::JDBCError: Key column 'domain' doesn't exist in table: CREATE UNIQUE INDEX `index_organizations_on_domain` ON `organizations` (`domain`)

Tasks: TOP => db:migrate

crowdmatt

# Here is some sample code to create a CloudFront distribution programmatically from Ruby.  
# Tested to be working on 06-23-2012.
# Amazon's documentation is wrong, so we're putting the XML here!
#
# We're using this at http://www.crowdmob.com/ to simplify our development process.
# 
# Prerequisite: You'll need the simple_aws gem which you can get from `gem install simple_aws`
#

require 'simple_aws/s3'

crowdmatt

  # Call me with linked_in_by_company('Disney')


  LINKEDIN = {
    api_key: ENV['LINKEDIN_API_KEY'],
    secret_key: ENV['LINKEDIN_SECRET_KEY'],
    consumers: {
      test_person: { 
        consumer_token: ENV['LINKEDIN_TESTPERSON_CONSUMER_TOKEN'],
        consumer_secret: ENV['LINKEDIN_TESTPERSON_CONSUMER_SECRET']

crowdmatt

{
"IAB1": "Arts & Entertainment",
"IAB1-1": "Books & Literature",
"IAB1-2": "Celebrity Fan/Gossip",
"IAB1-3": "Fine Art",
"IAB1-4": "Humor",
"IAB1-5": "Movies",
"IAB1-6": "Music",
"IAB1-7": "Television",
"IAB2": "Automotive",

crowdmatt

import (
  "bufio"
  "os"
)

// readWholeLine returns a single line (without the ending \n) from the input buffered reader.
// An error is returned iff there is an error with the buffered reader.
func readWholeLine(r *bufio.Reader) (string, error) {
  var (isPrefix bool = true
       err error = nil

crowdmatt

  <script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script><!-- required to getJSON from our server -->
  <script src="https://datejs.googlecode.com/files/date.js"></script><!-- required to parse dates in ISO format -->
  <script>
    var device_tracking_type = "mac_address";
    var current_user_device_type = "android";
    var current_user_country_code = "ca";
    var current_user_mac_address = "11:11:11:11";
    var current_user_location = "37.8188,-122.4784";
    var current_impression_uuid = "mynetworkimpression-iowd23e823jsd094";
    var current_time = new Date();

crowdmatt

Setting up Flume NG, listening to syslog over UDP, with an S3 Sink

My goal was to set up Flume on my web instances, and write all events into s3, so I could easily use other tools like Amazon Elastic Map Reduce, and Amazon Red Shift.

I didn't want to have to deal with log rotation myself, so I setup Flume to read from a syslog UDP source. In this case, Flume NG acts as a syslog server, so as long as Flume is running, my web application can simply write to it in syslog format on the specified port. Most languages have plugins for this.

At the time of this writing, I've been able to get Flume NG up and running on 3 ec2 instances, and all writing to the same bucket.

Install Flume NG on instances

crowdmatt

How to setup a Go / Golang WebServer with Kafka Log Aggregation to S3

This is a work in progress.

Install Kafka

Download the latest Kafka, which you can find at: http://kafka.apache.org/downloads.html

crowdmatt

openssl rsa -in MYFILE.pem -pubout > MYFILE.pub
ssh-keygen -f MYFILE.pub -i -m PKCS8

crowdmatt

Vagrant::Config.run do |config|
  config.vm.box = "squeeze64-ruby193"
  config.vm.box_url = "http://packages.diluvia.net/squeeze64-ruby193.box"
  config.vm.network :hostonly, "33.33.33.10"
  config.vm.share_folder "v-cookbooks", "/cookbooks", "."
end