Skip to content

Instantly share code, notes, and snippets.

View hamiltont's full-sized avatar
🕵️‍♂️
huntin' wabbits

Hamilton Turner hamiltont

🕵️‍♂️
huntin' wabbits
View GitHub Profile
@hamiltont
hamiltont / hadoop-MR-error.sh
Created February 27, 2014 21:18
An error running hadoop 2.2.0 mapreduce
# hadoop jar /share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar grep /input /ouput 'dfs[a-z.]+'
14/02/27 21:15:11 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/02/27 21:15:11 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
14/02/27 21:15:11 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job or Job#setJar(String).
14/02/27 21:15:11 INFO input.FileInputFormat: Total input paths to process : 23
14/02/27 21:15:11 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/tmp/hadoop-root/mapred/staging/root1084670987/.staging/job_local1084670987_0001
java.lang.NullPointerException
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getBlockIndex(FileInputFormat.java:390)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:365)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491)
@hamiltont
hamiltont / dock.py
Created April 17, 2014 03:15
Small example of docker remote api over ssh tunnel
import docker
import bgtunnel
import sys
def list_images(target_host):
print('Targeting %s' % target_host)
forwarder = bgtunnel.open(ssh_user="hamiltont",
silent=True,
ssh_address=target_host,
host_port=5555,
> # Setup data
> names <- c("Field 1","Field 2","Field 3")
> area <- c(1.3,1.4,1.1)
> soil <- factor(c("wet","dry","wet"))
> worms <- data.frame(names,area,soil)
> worms
names area soil
1 Field 1 1.3 wet
2 Field 2 1.4 dry
3 Field 3 1.1 wet
> # Using autocomplete here does nothing...
> plot(x,y,notc[Tab]
> # Using autocomplete here shows the notch option
> barplot(x,y,notc[Tab]
> barplot(x,y,notch=
===============FILE 1 - test.sh==================
#!/bin/bash
test_function () {
echo hi there
}
bash subshell.sh
===============FILE 2 - subshell.sh==================
#!/bin/bash -E
# If you are running a large number of installs back to back
# (e.g. for FwBm development), then setting this variable
# will cause apt-get and wget to use your proxy server. If
# your proxy server has a cache for static content, this can
# save you quite a lot of download time
# export http_proxy=http://10.0.1.0:3128
sudo apt-get -y install \
cmake build-essential automake `# Needed for building code` \
curl wget unzip `# Common tools` \
software-properties-common `# Needed for add-apt-repository` \
git-core mercurial `# Version control systems` \
libpcre3 libpcre3-dev libpcrecpp0 `# Regular expression support` \
libssl-dev libcurl4-openssl-dev `# SSL libraries` \
libmysqlclient-dev \
mongodb-clients \
libsqlite3-dev sqlite3 `# Database libraries` \
@hamiltont
hamiltont / console-debug.log
Last active August 29, 2015 14:04
Output of 1) console with DEBUG 2) console 3) test log file
╰─$ toolset/run-tests.py --test go --sleep 5 --type json --max-threads 8 --max-concurrency 8 --log DEBUG
FWROOT is /localhdd
run-tests : INFO Configuration: Namespace(client_host='10.0.2.17', client_identity_file='~/.ssh/id_rsa', client_user='hamiltont', conf_file='benchmark.cfg', database_host='10.0.2.18', database_identity_file='~/.ssh/id_rsa', database_os='linux', database_user='hamiltont', duration=15, exclude=None, install=None, install_error_action='continue', install_strategy='pertest', list_test_metadata=False, list_tests=False, log='DEBUG', max_concurrency=8, max_queries=20, max_threads=8, mode='benchmark', name='ec2', os='linux', parse=None, password_prompt=False, query_interval=5, server_host='10.0.2.3', sleep=5, starting_concurrency=8, test=['go'], type='json')
benchmarker : DEBUG latest_results_directory
benchmarker : INFO __gather_tests
benchmarker : INFO run
benchmarker : INFO __gather_tests
================================================================================
if benchmarker is None:
default_config = setup_util.get_fwroot() + "/benchmark.cfg.example"
config = ConfigParser.SafeConfigParser()
config.read(default_config)
defaults = dict(config.items("Defaults"))
benchmarker = Benchmarker(defaults)
class attrExample(d:
def __getattr__(self, item):
"""
Returns the attr if it's part of attrExample, otherwise
return the attr from foo
"""
try:
return self.item
except KeyError: