Skip to content

Instantly share code, notes, and snippets.

View rberger's full-sized avatar

Robert J. Berger rberger

68 followers · 133 following
View GitHub Profile
@rberger
rberger / knife.rb
Created February 18, 2011 08:14 — forked from jbz/knife.rb
# This knife.rb orig. provided by jtimberman@opscode.com - but any mistakes are mine!
current_dir = File.dirname(__FILE__)
platform = "https://api.opscode.com/organizations"
require 'grit'
branch = Grit::Repo.new("#{current_dir}/..").head.name
case branch
@rberger
rberger / Error from cascading.hbase
Created March 15, 2011 00:17
When I use cascading.hbase with current cascading-1.2.1
$ /mnt/hadoop/bin/hadoop jar test_cascading_hbase.jar data/small.txt
11/03/15 00:13:29 INFO flow.MultiMapReducePlanner: using application jar: /home/rberger/test_cascading_hbase.jar
11/03/15 00:13:29 INFO cascade.Cascade: Concurrent, Inc - Cascading 1.2.1 [hadoop-0.19.2+]
11/03/15 00:13:29 INFO flow.Flow: [insert] starting
11/03/15 00:13:29 INFO flow.Flow: [insert] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["data/small.txt"]"]
11/03/15 00:13:29 INFO flow.Flow: [insert] sink: cascading.hbase.HBaseTap@b6541340
11/03/15 00:13:30 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
11/03/15 00:13:30 INFO zookeeper.ZooKeeper: Client environment:host.name=ip-10-160-127-175.us-west-1.compute.internal
11/03/15 00:13:30 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_15
11/03/15 00:13:30 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
@rberger
rberger / Proposed HBase Backup using HBASE-1684
Created March 15, 2011 21:37
I'm in the midst of trying to wrangle an HBase backup/restore to/from S3 or HDFS
built around export/backup of 1 table at a time
using org.apache.hadoop.hbase.mapreduce.Export from HBASE-1684.
Just a reminder:
Usage: Export <tablename> <outputdir> [<versions> [<starttime> [<endtime>]]]
In the psuedo code below:
persistant_store is some kind of non-HBase store in the Cloud that you can just
@rberger
rberger / gist:941414
Created April 25, 2011 22:41
Chef problem with application
[Mon, 25 Apr 2011 22:40:38 +0000] INFO: Starting Chef Run (Version 0.9.16)
[Mon, 25 Apr 2011 22:40:44 +0000] INFO: Ran execute[apt-get update] successfully
[Mon, 25 Apr 2011 22:40:45 +0000] INFO: Already deployed app at /mnt/app/runadotcom/releases/5aa03ede0914be395442c2beb527a597899a0dfd, forcing.
[Mon, 25 Apr 2011 22:40:46 +0000] INFO: deploying branch: master
[Mon, 25 Apr 2011 22:40:46 +0000] INFO: ensuring proper ownership
[Mon, 25 Apr 2011 22:40:46 +0000] INFO: updating the cached checkout
[Mon, 25 Apr 2011 22:40:46 +0000] INFO: copying the cached checkout to /mnt/app/runadotcom/releases/5aa03ede0914be395442c2beb527a597899a0dfd
[Mon, 25 Apr 2011 22:40:46 +0000] INFO: ensuring proper ownership
[Mon, 25 Apr 2011 22:40:46 +0000] INFO: Running callback before_migrate code block
[Mon, 25 Apr 2011 22:40:46 +0000] INFO: Ran execute[(ln -s ../../../shared/database.yml config/database.yml && rake gems:install); rm config/database.yml] successfully
@rberger
rberger / gist:962752
Created May 9, 2011 15:42
Migration fail during chef run
[Mon, 09 May 2011 09:01:26 +0000] DEBUG: Executing rake db:migrate RAILS_ENV=testing
[Mon, 09 May 2011 09:01:32 +0000] DEBUG: ---- Begin output of rake db:migrate RAILS_ENV=testing ----
[Mon, 09 May 2011 09:01:32 +0000] DEBUG: STDOUT: (in /mnt/app/runa_dashboard/releases/e8469b1c251a7b2db1b4b410499f11ac7d6c1958)
== OneBigMigrationFile: migrating ============================================
-- create_table(:audits, {:force=>true})
-> 0.0237s
-- add_index(:audits, [:auditable_id, :auditable_type], {:name=>"auditable_index"})
-> 0.0227s
-- add_index(:audits, [:auditable_parent_id, :auditable_parent_type], {:name=>"auditable_parent_index"})
-> 0.0228s
@rberger
rberger / gist:1015398
Created June 8, 2011 21:05
Error Building native extensions for dep_selector-0.0.8 in chef 0.10 install
We couldn’t find that file to show.
@rberger
rberger / gist:1015399
Created June 8, 2011 21:05
Error Building native extensions for dep_selector-0.0.8 in chef 0.10 install
We couldn’t find that file to show.
@rberger
rberger / gist:1015626
Created June 8, 2011 22:40
Problem starting chef-server on Ubuntu 9.10
Note: The file /usr/lib/ruby/gems/1.8/gems/dep_selector-0.0.8/lib/dep_gecode.so does exisit:
# ls -l /usr/lib/ruby/gems/1.8/gems/dep_selector-0.0.8/lib/dep_gecode.so
-rwxr-xr-x 1 root root 478265 2011-06-08 22:14 /usr/lib/ruby/gems/1.8/gems/dep_selector-0.0.8/lib/dep_gecode.so
# file /usr/lib/ruby/gems/1.8/gems/dep_selector-0.0.8/lib/dep_gecode.so
/usr/lib/ruby/gems/1.8/gems/dep_selector-0.0.8/lib/dep_gecode.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
2011-06-08_22:36:22.30058 Merb root at: /usr/lib/ruby/gems/1.8/gems/chef-server-api-0.10.0
2011-06-08_22:36:22.30063 ~ No Gemfile found! If you're generating new app with merb-gen this is fine, otherwise run: bundle init to create Gemfile
2011-06-08_22:36:22.30064 Loading init file from ./config/init.rb
@rberger
rberger / nginx_unicorn.rb
Created June 15, 2011 22:06 — forked from threetee/nginx_unicorn.rb
chef nginx_unicorn recipe (part of modified application cookbook)
app = node.run_state[:current_app]
#unicorn
include_recipe "application::unicorn"
#nginx
include_recipe "nginx::source"
#nginx/unicorn proxy
socket = "/tmp/unicorn_#{app['id']}.sock"
@rberger
rberger / recipe.rb
Created June 15, 2011 22:28 — forked from peplin/recipe.rb
S3 File Resource for Chef
# Source accepts the protocol s3:// with the host as the bucket
# access_key_id and secret_access_key are just that
s3_file "/var/bulk/the_file.tar.gz" do
source "s3://your.bucket/the_file.tar.gz"
access_key_id your_key
secret_access_key your_secret
owner "root"
group "root"
mode 0644
end