Skip to content

Instantly share code, notes, and snippets.

Avatar

Philip (flip) Kromer mrflip

View GitHub Profile
View gist:e3f3999af39bb9986475
14/11/19 10:08:12 INFO listeners.LipstickPPNL: --- Init TBPPNL ---
2014-11-19 10:08:12,865 [main] INFO com.netflix.lipstick.Main - build version: 0.6-SNAPSHOT, ts: 2014-11-19T09:02Z, git: 6324e130b4df844930f33cc7fb08aed94d07e76e
2014-11-19 10:08:12,865 [main] INFO com.netflix.lipstick.Main - Logging error messages to: /home/chimpy/book/code/pig_1416391692863.log
2014-11-19 10:08:13,245 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/chimpy/.pigbootup not found
2014-11-19 10:08:13,740 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-11-19 10:08:13,741 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://nn:8020
2014-11-19 10:08:14,215 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: jt:8021
2014-11-19 10:08:14,273 [main] INFO org.apache.hadoop.conf.Configuration.deprecati
View gist:94b460f1db531de7e165
sudo -u hdfs hadoop fs -mkdir -p \
/tmp /tmp/mapred/system \
/user/root /user/chimpy \
$HADOOP_LOG_DIR/yarn-apps \
$HADOOP_BULK_DIR/yarn-staging/history/done_intermediate \
/var/lib/hadoop-hdfs/cache/mapred/mapred/staging \
/user/hive/warehouse
sudo -u hdfs hadoop fs -chmod -R 1777 \
/tmp /tmp/mapred/system \
@mrflip
mrflip / Gemfile
Last active Aug 29, 2015
Spike of a Roomie Remote API
View Gemfile
source 'https://rubygems.org'
gem 'gorillib', "~> 0.6"
gem 'pry', "~> 0.10"
gem 'multi_json', ">= 1.1"
gem 'crack'
gem 'erubis'
@mrflip
mrflip / README.md
Last active Aug 29, 2015
Roomie Remote Image Remotes borken
View README.md

I am trying to make a custom image remote. Roomie crashes even on a trivial modification to the DDK example.

I've tried a whole lot of things, but here I'll describe the most minimal test case I can conceive.

First, I reset my enviroment back to zero:

  • I first reset my configuration
  • Disabled Dropbox polling and Wifi sync
  • Removed ~/Dropbox/Roomie
@mrflip
mrflip / README.md
Last active Aug 29, 2015
Single-node Hadoop cluster
View README.md

In the below, we'll use the following to track the version of the image you'll use

dv=1

Launching image

docker run -it --name hadoop_local -u chimpy -w /home/chimpy -e HOME=/home/chimpy -e TERM="$TERM" "bd4c/hadoop_local:v1.$dv" /bin/bash -l
View optional args in lambdas.rb
# this works
a = lambda{ |b| b*3 }
# => #<Proc:0x01f5f540@(irb):125>
a.call 4
# => 12
# strangely, this does too (with a warning)
a.call 4, 6
# (irb):125: warning: multiple values for a block parameter (2 for 1)
# from (irb):131
@mrflip
mrflip / gist:7983
Created Aug 29, 2008
Tab complete the 'open' command in OSX
View gist:7983
# http://woss.name/2005/09/06/bash-completion-for-mac-os-x-open/
if [ "`uname`" = "Darwin" ]; then
_open() {
local cur prev
COMPREPLY=()
cur=${COMP_WORDS[COMP_CWORD]}
prev=${COMP_WORDS[COMP_CWORD-1]}
if [ "${prev}" = -a ]; then
local IFS=$'\n'
View L_project.sh
== An elementary as hell solution to stephenfry's #L challenge: ==
-- Get the twitter scrape from infochimps.org
-- http://blog.infochimps.org/2008/12/29/massive-scrape-of-twitters-friend-graph/
-- (not available yet, but sometime soon)
--
-- Then run this pig (http://wiki.apache.org/pig/) code
-- to extract all twitter screen names with three or more L's
-- It's not properly case insensitive but who cares.
--
View gist:50411
#
# Words by L-fraction, from lull to zill to algebraically
#
# All words valid in US Scrabble that are
# composed of 21% or more L's (including space),
# sorted by descending L-fraction. For help in
# @stephenfry's http://hashtags.org/tag/l
# "L challenge"
#
View Nested Model Forms
Nested Model Forms
Posted by michael January 26, 2009 @ 08:55 AM
The most popular request on our new Feedback site was for the ability to easily manage multiple models in a single form. Thankfully Eloy Duran has a patch that does just this. But before we roll it into rails 2.3 we want to get some more feedback from you guys. Eloy's written this brief introduction to the patch, so take a look, and add any feedback you have to the lighthouse ticket.
In Rails it has always been a little tricky to create a single form for both a model and its associations. Last summer, the :accessible option was committed as a first stab at a unified solution for dealing with nested models. However, this feature was pulled before the 2.2 release because it only supported nested models during object creation.
The resulting discussion on the core mailing list and our need for mass assignment have resulted in this patch for ticket #1202 which is now up for scrutinizing.