Skip to content

Instantly share code, notes, and snippets.

@damianknopp
damianknopp / logGrep1.sh
Created March 14, 2014 15:22
cat, grep, truncate, more
cat log.txt | grep EXPRESSION | sed 's/.*EXPRESSION://g' - | more
@damianknopp
damianknopp / SolrIndexer.java
Created March 18, 2014 23:02
index into solr, and quickly please
package dmk.solr;
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.UUID;
db.collection.find({ principal: /user1/, deleted: false, settings: { $ne: undefined } }, { principal:1, "_id":false })
db.collection( { "_id": "" }, { $set: { "deleted": true }})
db.collection( { principal: /user1/ }).count()
db.collection( { principal: /user1/ }, "deleted": { $exists: false }).count()
db.collection( { principal: /.*/ }, $set: { delete: true }, { upsert: true, multi: true })
@damianknopp
damianknopp / createPem.sh
Created March 24, 2014 17:16
create PEM
#!/bin/bash
if [ -z "$1"] || [ -z "$2" ]
then
echo "Usage: `basename $0 <certname> <user>"
fi
openssl pkcs12 -in $1 -nomacver -out $2.pem
#!/bin/bash
openssl x509 -in $1.pem -text
@damianknopp
damianknopp / createMongoPEM.sh
Created March 24, 2014 17:21
creating self signed cert to use with mongodb
#!/bin/bash
#sample take from mongodb docs, configure ssl
# the below generates a self signed cert and private key w/ no pass valid for 365 days
CRT=mongodb-cert.crt
KEY=mongodb-cert.key
openssl req -new -x509 -days 365 -nodes -out $CRT -keyout $KEY
cat $CRT $KEY > mongodb.pem
@damianknopp
damianknopp / runMapperWithLibJars.sh
Created March 28, 2014 18:07
hadoop .20x series with libjars
#!/bin/bash
IN_DIR=/home/hadoop/dmk-in
OUT_DIR=/home/hadoop/dmk-out
LIB=$PWD/lib
CLASSPATH="dmk-mapper.jar"
MY_HADOOP_CP="-libjars $PWD/dmk-mapper.jar"
LIB_JAR_DELIM=,
@damianknopp
damianknopp / TestJob.java
Created March 28, 2014 18:26
Simple Hadoop .20x Job called with libjars using ToolRunner
package dmk;
public class TestJob extends Configured implements Tool{
private static final String JAVA_OPTS = "-Xmx500m -Xms500m";
private static final String USAGE= "Usage: hadoop jar <jarfile> TestJob [generic opts] <input> <output>";
public in run(String[] args){
Configuration conf = getConf();
String[] otherArgs = new GernicOptsionParser(conf, args).getRemainingArgs();
conf.set("mapred.child.java.opts", JAVA_OPTS);
@damianknopp
damianknopp / mongo export
Created April 17, 2014 21:56
mongodb export records
#!/bin/bash
USERNAME=
HOSTNAME=
DATABASENAME=
COLLECTIONNAME=
#export mongodb records
mongoexport --ssl -u $USERNAME -h $HOSTNAME -d $DATABASENAME -c $COLLECTIONAME -p > records.dump.`date`.out
@damianknopp
damianknopp / iptables.sh
Created June 13, 2014 16:24
restrict to tunnel only
iptables -A INPUT-p tcp -s localhost --dport 8983 -j ACCEPT
iptables -A INPUT -p tcp --dport 8983 -j DROP