Skip to content

Instantly share code, notes, and snippets.

View kitplummer's full-sized avatar
🏠
Working from home

Kit Plummer kitplummer

🏠
Working from home
View GitHub Profile
@kitplummer
kitplummer / emr-spark-launch.sh
Created November 18, 2015 15:20
Spark EMR Cluster script
# elastic-mapreduce --create --name "Spark Example Project" --instance-type m1.xlarge --instance-count 3 \
# --jobflow-role SparkRole --service-role EMR_DefaultRole \
# --bootstrap-action s3://support.elasticmapreduce/spark/install-spark --bootstrap-name "Install Spark" \
# --jar s3://elasticmapreduce/libs/script-runner/script-runner.jar --step-name "Run Spark Example Project" \
# --step-action TERMINATE_JOB_FLOW \
# --arg s3://snowplow-hosted-assets/common/spark/run-spark-job-0.1.0.sh \
# --arg s3://sparkcluster/spark-example-project-0.2.0.jar \
# --arg com.snowplowanalytics.spark.WordCountJob \
# --arg s3n://sparkcluster/hello.txt \
# --arg s3n://sparkcluster/results
@kitplummer
kitplummer / openstack.groovy
Created June 12, 2013 22:01
Simple Groovy script working with OpenStack's Keystone and Compute(Nova) APIs.
@Grab(group="org.codehaus.groovy.modules.http-builder", module="http-builder", version='0.5.2')
import groovyx.net.http.RESTClient
import groovy.util.slurpersupport.GPathResult
import groovy.json.*
keystone = new RESTClient( 'http://10.0.1.49:5000/v2.0/' )
resp = keystone.post( path : 'tokens',
body : [auth:[passwordCredentials:[username: "admin", password:"stack"], tenantId: "2ba2d60c5e8d4d1b86549d988131fe48"]],
import groovyx.net.http.HTTPBuilder
import static groovyx.net.http.Method.GET
import static groovyx.net.http.ContentType.TEXT
def http = new HTTPBuilder( 'https://scapsync.com' )
def json = ""
http.request(GET,TEXT) { req ->
uri.path = '/search' // overrides any path in the default URL
➜ ThreadFix sh threadfix.sh start
Looking for a key in /Users/kitplummer/Code/ThreadFix/tomcat/keystore
Using pre-generated keystore.
mkdir: ./tomcat/logs: File exists
Using CATALINA_BASE: ./tomcat
Using CATALINA_HOME: ./tomcat
Using CATALINA_TMPDIR: ./tomcat/temp
Using JRE_HOME: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Using CLASSPATH: ./tomcat/bin/bootstrap.jar:./tomcat/bin/tomcat-juli.jar
Oct 8, 2013 8:37:59 AM org.apache.coyote.AbstractProtocol stop
➜ ThreadFix sh threadfix.sh start
Looking for a key in /Users/kitplummer/Code/ThreadFix/tomcat/keystore
Using pre-generated keystore.
mkdir: ./tomcat/logs: File exists
Using CATALINA_BASE: ./tomcat
Using CATALINA_HOME: ./tomcat
Using CATALINA_TMPDIR: ./tomcat/temp
Using JRE_HOME: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Using CLASSPATH: ./tomcat/bin/bootstrap.jar:./tomcat/bin/tomcat-juli.jar
Oct 8, 2013 8:40:52 AM org.apache.coyote.AbstractProtocol stop
[root@ddf2 ~]# puppet agent --test --no-daemon
Warning: Unable to fetch my node definition, but the agent run will continue:
Warning: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA generated on stackbox at 2013-07-11 14:30:29 -0700]
Info: Retrieving plugin
Error: /File[/var/opt/lib/pe-puppet/lib]: Failed to generate additional resources using 'eval_generate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA generated on stackbox at 2013-07-11 14:30:29 -0700]
Error: /File[/var/opt/lib/pe-puppet/lib]: Could not evaluate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA generated on stackbox at 2013-07-11 14:30:29 -0700] Could not retrieve file metadata for puppet://puppet/plugins
$opensearch_sources = ["ddf1", "ddf2"]
each($opensearch_sources) |$opensearch_source| {
file { "/usr/local/${package}-${version}/deploy/OpenSearchSource.${$opensearch_source}.cfg":
mode => "0644",
group => $ddf_user,
owner => $ddf_user,
content => template("ddf/OpenSearchSource.cfg.erb"),
require => File["/usr/local/${package}-${version}"]
}
}
err: /Stage[main]/Omar::Omar/Package[postgis2_92]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install postgis2_92' returned 1: Error: Package: postgis2_92-2.0.3-2.rhel6.x86_64 (pgdg92)
Requires: libgdal.so.1()(64bit)
Available: gdal-1.7.2-1.el6.x86_64 (pgdg91)
libgdal.so.1()(64bit)
Available: gdal-1.7.3-15.el6.x86_64 (epel)
libgdal.so.1()(64bit)
Installed: gdal-1.9.2-1.x86_64 (@codice)
Not found
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
@kitplummer
kitplummer / initialize.sh
Created February 1, 2013 00:10
initialize openstack instance.
#!/bin/bash
if [ -f /root/.initialized ]; then
exit 0
fi
FQDN=`/usr/bin/curl -s http://169.254.169.254/latest/user-data | grep hostname | sed 's/.*=//' | sed 's/ //'`
HOSTNAME=`echo ${FQDN} | awk -F"." '{print $1}'`
PM=`/usr/bin/curl -s http://169.254.169.254/latest/user-data | grep puppetmaster | sed 's/.*=//' | sed 's/ //'`