Skip to content

Instantly share code, notes, and snippets.

View medined's full-sized avatar

David Medinets medined

41 followers · 19 following
View GitHub Profile
@medined
medined / gist:118992
Created May 28, 2009 00:05
Book Model Generation for ActiveScaffold Presentation
script/generate model book person_id:integer name:string pages:integer price:float invoiced_at:datetime purchased_on:date paid_at:timestamp checkin:time description:text paperback:boolean
@medined
medined / accumulo_notes
Created December 29, 2011 19:04
Accumulo Installation and Configuration Steps on a Ubuntu VirtualBox Instance
Oct 17, 2012 - See https://github.com/medined/accumulo_stackscript for an even better script. Really ignore the stuff below. Go. Scoot.
Aug 28, 2012 - See http://affy.blogspot.com/2012/08/stackscript-for-accumulo-on-linode.html for a more concise method is configuring accumulo. I'll leave this gist unchanged for fans of history.
My goal was to get Accumulo running on a VirtualBox Ubuntu instance. I was successful using the following steps. If a line starts with $ then it is a command-line to execute. Note that you'll need to have sudo privilege. My username was 'ubuntu'. If you are using a different username, you'll need to change the process a little bit. I'll try to point out where.
https://issues.apache.org/jira/browse/ACCUMULO
##########
# Start a new VirtualBox instance using the Ubuntu 11.10
@medined
medined / install_accumulo_in_home_directory.sh
Created April 24, 2013 03:05
how to install accumulo (and everything else) to your home directory
#!/bin/bash
export HADOOP_VERSION=hadoop-1.0.4
export CDIR=`pwd`
export LOGFILE=~/build.log
export PASSWORD=`openssl passwd -1 password`
##########
# enable logging. Logs both to file and screen.
exec 2>&1
@medined
medined / 01_make_image
Created July 9, 2014 16:32
david@zareason-verix545:~/projects/docker-accumulo$ ./make_image.sh
Installing nsenter to /target
Installing docker-enter to /target
Warning: '-rm' is deprecated, it will be replaced by '--rm' soon. See usage.
Sending build context to Docker daemon 165.9 kB
Sending build context to Docker daemon
Step 0 : FROM sroegner/doop
---> eb64b59ce2a4
Step 1 : RUN curl -L http://apache.osuosl.org/accumulo/1.5.1/accumulo-1.5.1-bin.tar.gz -o /tmp/acc.tgz && tar xzf /tmp/acc.tgz -C /usr/lib && rm -vf /tmp/acc.tgz && id
---> Using cache
@medined
medined / 02_file_diff
Created July 9, 2014 16:34
david@zareason-verix545:~/projects/docker-accumulo$ docker diff grail
C /data1
C /data1/hdfs
C /data1/hdfs/dn
C /data1/hdfs/dn/current
C /data1/hdfs/dn/current/BP-1274135865-172.17.0.10-1404767453280
C /data1/hdfs/dn/current/BP-1274135865-172.17.0.10-1404767453280/current
C /data1/hdfs/dn/current/BP-1274135865-172.17.0.10-1404767453280/current/VERSION
C /data1/hdfs/dn/current/BP-1274135865-172.17.0.10-1404767453280/current/rbw
A /data1/hdfs/dn/current/BP-1274135865-172.17.0.10-1404767453280/current/rbw/blk_1073741826
@medined
medined / 03_namenode_log
Created July 9, 2014 16:37
-bash-4.1# cat /var/log/supervisor/namenode-stderr---supervisor-Ggf9Oz.log
14/07/09 12:33:42 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = grail/172.17.0.40
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.4.0.2.1.2.1-471
STARTUP_MSG: classpath = /etc/hadoop/conf:/usr/lib/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/usr/lib/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/lib/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/lib/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/usr/lib/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/lib/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/lib/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/lib/hadoop/share/hadoop/common/lib
@medined
medined / 04_datanode_log
Created July 9, 2014 16:39
-bash-4.1# cat datanode-stderr---supervisor-cq7diO.log
14/07/09 12:33:42 INFO datanode.DataNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG: host = grail/172.17.0.40
STARTUP_MSG: args = []
STARTUP_MSG: version = 2.4.0.2.1.2.1-471
STARTUP_MSG: classpath = /etc/hadoop/conf:/usr/lib/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/usr/lib/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/lib/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/lib/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/lib/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/usr/lib/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/lib/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/lib/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/lib/hadoop/share/hadoop/common/lib/microsoft-windowsaz
@medined
medined / 05_accumulo_gc_log
Created July 9, 2014 16:40
-bash-4.1# cat accumulo-gc-stderr---supervisor-6DKWec.log
Thread "gc" died java.lang.reflect.InvocationTargetException
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.accumulo.start.Main$1.run(Main.java:103)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.RuntimeException: Can't tell if Accumulo is initialized; can't read instance id at /accumulo/instance_id
@medined
medined / tserver_grail.debug.log
Created July 26, 2014 15:12
TSERVER showing MYHOSTNAME (which should be grail)
2014-07-26 10:14:59,767 [server.Accumulo] INFO : tserver starting
2014-07-26 10:14:59,769 [server.Accumulo] INFO : Instance c4334106-5e4f-4807-8fb4-df30fd244b3f
2014-07-26 10:14:59,771 [server.Accumulo] INFO : Data Version 6
2014-07-26 10:14:59,771 [server.Accumulo] INFO : Attempting to talk to zookeeper
2014-07-26 10:14:59,774 [zookeeper.ZooKeeper] INFO : Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2014-07-26 10:14:59,774 [zookeeper.ZooKeeper] INFO : Client environment:host.name=grail
2014-07-26 10:14:59,774 [zookeeper.ZooKeeper] INFO : Client environment:java.version=1.7.0_55
2014-07-26 10:14:59,774 [zookeeper.ZooKeeper] INFO : Client environment:java.vendor=Oracle Corporation
2014-07-26 10:14:59,774 [zookeeper.ZooKeeper] INFO : Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.55.x86_64/jre
2014-07-26 10:14:59,775 [zookeeper.ZooKeeper] INFO : Client environment:java.class.path=/usr/lib/accumulo-1.6.0/conf:/usr/lib/accumulo/lib/accumulo-start.jar:/usr/l
@medined
medined / adjacency_matrices
Created December 3, 2014 05:08
Adjacency Matrices
Adjacency matrices are often square and both out-vertices (rows) and the in-vertices (columns) are the same set of vertices.
A B C
D 1 0 0
F 0 0 0
G 0 0 1
The above matrix represents two 6 vertices and two edges. They could be represented as:
D -> A