Skip to content

Instantly share code, notes, and snippets.

View zcox's full-sized avatar

Zach Cox zcox

52 followers · 7 following
View GitHub Profile
@zcox
zcox / gist:37245d41825633c9b8b4
Created April 10, 2015 22:28
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/druid-0.7.1.1/lib/log4j-slf4j-impl-2.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2015-04-10T22:24:38,462 INFO [main] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2015-04-10T22:24:38,519 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final
2015-04-10T22:24:39,206 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:postgresql-metadata-storage, io.druid.extensions:druid-hdfs-storage], defaultVersion='0.7.1.1', localRepository='/opt/druid/repository', remot
@zcox
zcox / gist:01e8abfe0922b49ea55a
Created April 10, 2015 19:40
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/druid-0.7.1.1/lib/log4j-slf4j-impl-2.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2015-04-10T19:32:19,855 INFO [main] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2015-04-10T19:32:19,916 INFO [main] org.hibernate.validator.internal.util.Version - HV000001: Hibernate Validator 5.1.3.Final
2015-04-10T19:32:20,587 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:postgresql-metadata-storage, io.druid.extensions:druid-hdfs-storage], defaultVersion='0.7.1.1', localRepository='/opt/druid/repository', remot
@zcox
zcox / gist:bc2861977509cb111683
Created April 10, 2015 17:22
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/druid-0.7.0/lib/log4j-slf4j-impl-2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2015-04-10T16:56:41,706 INFO [main] io.druid.guice.PropertiesModule - Loading properties from common.runtime.properties
2015-04-10T16:56:42,623 INFO [main] io.druid.guice.JsonConfigurator - Loaded class[class io.druid.guice.ExtensionsConfig] from props[druid.extensions.] as [ExtensionsConfig{searchCurrentClassloader=true, coordinates=[io.druid.extensions:postgresql-metadata-storage, io.druid.extensions:druid-hdfs-storage], defaultVersion='0.7.0', localRepository='/opt/druid/repository', remoteRepositories=[http://repo1.maven.org/maven2/, https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local]}]
2015-04-10T1
@zcox
zcox / Main.scala
Created March 5, 2015 20:47
package com.banno
import org.apache.curator.framework.CuratorFrameworkFactory
import org.apache.curator.retry.ExponentialBackoffRetry
import com.metamx.tranquility.druid.{DruidBeams, DruidLocation, DruidRollup, SpecificDruidDimensions}
import com.metamx.tranquility.beam.ClusteredBeamTuning
import com.metamx.common.Granularity
import io.druid.query.aggregation.{CountAggregatorFactory, LongSumAggregatorFactory}
import io.druid.query.aggregation.hyperloglog.HyperUniquesAggregatorFactory
import io.druid.granularity.QueryGranularity
@zcox
zcox / build.sbt
Last active August 29, 2015 14:16
resolvers ++= Seq(
"bintray-banno-oss-releases" at "http://dl.bintray.com/banno/oss",
"Metamarkets" at "https://metamx.artifactoryonline.com/metamx/pub-libs-releases-local/",
"clojars" at "http://clojars.org/repo/")
libraryDependencies ++= Seq(
"org.apache.kafka" % "kafka-clients" % "0.8.2.0",
"org.twitter4j" % "twitter4j-stream" % "4.0.2",
"com.metamx" %% "tranquility" % "0.3.4",
"jsonz" %% "jsonz" % "1.0.0",
@zcox
zcox / create-task.sh
Created March 5, 2015 15:49
#!/bin/bash
JSON_FILE=$1
curl -X POST -H "Content-Type: application/json" -d @$JSON_FILE http://192.168.59.103:8085/druid/indexer/v1/task
@zcox
zcox / SamzaContainerShutdown.scala
Last active August 29, 2015 14:14
import org.apache.samza.container.{SamzaContainer, RunLoop}
import org.apache.samza.job.local.ThreadJob
import java.util.concurrent.atomic.AtomicBoolean
import org.slf4j.LoggerFactory
/** Utilities for clean Samza container shutdown. Note that these utilities depend on private, internal implementation details of various Samza components,
* accessed via Java reflection, and will not work if these details change in Samza releases after 0.8.0. See also https://issues.apache.org/jira/browse/SAMZA-506.
*
* If you have a SamzaContainer instance, you can just call addShutdownHook(container) and on jvm shutdown (i.e. SIGTERM) the container will be cleanly shut down.
*
@zcox
zcox / 1
Last active August 29, 2015 14:07
MBP, quad-core, 16GB RAM; 3 node local Mesos cluster via fig-mesos; 3 node Kafka cluster in Mesos; tested for fun using https://github.com/jamiealquiza/sangrenel
# bin/sangrenel -brokers="dev.banno.com:31131,dev.banno.com:31591,dev.banno.com:31750"
::: Sangrenel :::
Starting 1 workers
Message size 300 bytes
client_1 connected
2014-10-17T18:25:12Z Generating 1Mb/sec @ 637 messages/sec | topic: sangrenel | 27.63ms avg latency
2014-10-17T18:25:17Z Generating 2Mb/sec @ 854 messages/sec | topic: sangrenel | 16.61ms avg latency
2014-10-17T18:25:22Z Generating 2Mb/sec @ 974 messages/sec | topic: sangrenel | 13.88ms avg latency
@zcox
zcox / gist:0f2b260d29e18d40d038
Created October 6, 2014 18:24
2014-10-06 18:23:49,429 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler: Trying to fulfill reservation for application appattempt_1412618138668_0005_000001 on node: precise64:35969
2014-10-06 18:23:49,429 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AppSchedulable: Node offered to app: application_1412618138668_0005 reserved: true
2014-10-06 18:23:49,429 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AppSchedulable: Making reservation: node=precise64 app_id=application_1412618138668_0005
2014-10-06 18:23:49,430 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerApp: Application application_1412618138668_0005 reserved container container_1412618138668_0005_01_000001 on node host: precise64:35969 #containers=8 available=<memory:4096, vCores:0> used=<memory:4096, vCores:8>, currently has 1 at priority 0; currentReservation 512
2014-10-06 18:23:49,430 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.
@zcox
zcox / gist:3d3c7e4dd72c384f716a
Created August 15, 2014 20:28
Solarized Light sidebar theme for Slack
#FDF6E3,#EEE8D5,#EEE8D5,#586E75,#EEE8D5,#657B83,#859900,#CB4B16