Skip to content

Instantly share code, notes, and snippets.

View sixtus's full-sized avatar

Hagen Rother sixtus

30 followers · 12 following
View GitHub Profile
@sixtus
sixtus / check_druid_segments.rb
Created November 18, 2015 09:54
#!/usr/bin/env ruby
require 'nagios'
require 'json'
require 'stringio'
require 'zk'
require 'zlib'
require 'time'
require 'yaml'
@sixtus
sixtus / gist:10124853
Created April 8, 2014 13:34
com.metamx.common.RE: Failure on row[{... cut ... }]
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:81)
at io.druid.indexer.HadoopDruidIndexerMapper.map(HadoopDruidIndexerMapper.java:33)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
@sixtus
sixtus / gist:8437425
Created January 15, 2014 14:39
2014-01-15 14:36:33,845 INFO [main] io.druid.indexer.IndexGeneratorJob - Job metrik-index-generator-[2014-01-15T09:00:00.000Z/2014-01-15T10:00:00.000Z] submitted, status available at http://hadoop2.madvertise.net:50030/jobdetails.jsp?jobid=job_201401151459_0037
2014-01-15 14:36:33,845 INFO [main] org.apache.hadoop.mapred.JobClient - Running job: job_201401151459_0037
2014-01-15 14:36:33,846 INFO [main] org.apache.hadoop.mapred.JobClient - Job complete: job_201401151459_0019
2014-01-15 14:36:33,855 INFO [main] org.apache.hadoop.mapred.JobClient - Counters: 29
2014-01-15 14:36:33,855 INFO [main] org.apache.hadoop.mapred.JobClient - Job Counters
2014-01-15 14:36:33,855 INFO [main] org.apache.hadoop.mapred.JobClient - Launched reduce tasks=1
2014-01-15 14:36:33,855 INFO [main] org.apache.hadoop.mapred.JobClient - SLOTS_MILLIS_MAPS=62605
2014-01-15 14:36:33,855 INFO [main] org.apache.hadoop.mapred.JobClient - Total time spent by all reduces waiting after reserving slots (ms)=0
2014-01-15 14:36:33,855
@sixtus
sixtus / gist:8019688
Created December 18, 2013 09:37
strange druid 0.6.37 historical log
Dec 18 10:34:26 fh1 druid-coordinator[18527]: Dec 18, 2013 9:34:26 AM com.sun.jersey.spi.container.ContainerResponse write
Dec 18 10:34:26 fh1 druid-coordinator[18527]: SEVERE: The registered message body writers compatible with the MIME media type are:
Dec 18 10:34:26 fh1 druid-coordinator[18527]: */* ->
Dec 18 10:34:26 fh1 druid-coordinator[18527]: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider
Dec 18 10:34:26 fh1 druid-coordinator[18527]: Dec 18, 2013 9:34:26 AM com.sun.jersey.spi.container.ContainerResponse logException
Dec 18 10:34:26 fh1 druid-coordinator[18527]: SEVERE: Mapped exception to response: 500 (Internal Server Error)
Dec 18 10:34:26 fh1 druid-coordinator[18527]: javax.ws.rs.WebApplicationException: com.sun.jersey.api.MessageException: A message body writer for Java class java.lang.String, and Java type class java.lang.String, and MIME media type application/json was not found
Dec 18 10:34:26 fh1 druid-coordinator[18527]: at com.sun.jersey.spi.container.ContainerResponse.write(ContainerR
@sixtus
sixtus / gist:7988894
Created December 16, 2013 15:32
druid batch import failing on 0.6.30
attempt_201311181919_17314_r_000000_2: java.lang.NullPointerException
attempt_201311181919_17314_r_000000_2: at io.druid.segment.incremental.IncrementalIndex.add(IncrementalIndex.java:188)
attempt_201311181919_17314_r_000000_2: at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:289)
attempt_201311181919_17314_r_000000_2: at io.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:237)
attempt_201311181919_17314_r_000000_2: at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
attempt_201311181919_17314_r_000000_2: at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:650)
attempt_201311181919_17314_r_000000_2: at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
attempt_201311181919_17314_r_000000_2: at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
attempt_201311181919_17314_r_000000_2: at java.security.AccessController.doPrivileged(Native Method)
attempt_201311181919_17314_r_000000_2: at java
@sixtus
sixtus / gist:4985125
Created February 19, 2013 11:44
HTTP ERROR 500
Problem accessing /druid/v2/datasources/events. Reason:
INTERNAL_SERVER_ERROR
Caused by:
java.lang.NullPointerException
at com.metamx.druid.http.ClientInfoResource.getDatasourceDimensions(ClientInfoResource.java:106)
at com.metamx.druid.http.ClientInfoResource.getDatasource(ClientInfoResource.java:86)
@sixtus
sixtus / gist:4738937
Created February 8, 2013 13:04
attempt_201301291408_2900_r_000000_0: java.lang.NullPointerException
attempt_201301291408_2900_r_000000_0: at com.metamx.druid.index.v1.QueryableIndexIndexableAdapter$2$1.next(QueryableIndexIndexableAdapter.java:214)
attempt_201301291408_2900_r_000000_0: at com.metamx.druid.index.v1.QueryableIndexIndexableAdapter$2$1.next(QueryableIndexIndexableAdapter.java:152)
attempt_201301291408_2900_r_000000_0: at com.google.common.collect.Iterators$8.next(Iterators.java:811)
attempt_201301291408_2900_r_000000_0: at com.google.common.collect.Iterators$8.next(Iterators.java:811)
attempt_201301291408_2900_r_000000_0: at com.google.common.collect.Iterators$PeekingImpl.peek(Iterators.java:1186)
attempt_201301291408_2900_r_000000_0: at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:46)
attempt_201301291408_2900_r_000000_0: at com.metamx.common.guava.MergeIterator$1.compare(MergeIterator.java:42)
attempt_201301291408_2900_r_000000_0: at java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.ja
@sixtus
sixtus / gist:4738718
Created February 8, 2013 12:29
attempt_201301291408_2896_r_000000_1: 2013-02-08 12:28:05,687 WARN [main] org.apache.hadoop.mapred.Child - Error running child
attempt_201301291408_2896_r_000000_1: java.lang.NullPointerException
attempt_201301291408_2896_r_000000_1: at com.metamx.druid.index.column.SimpleColumn.getDictionaryEncoding(SimpleColumn.java:75)
attempt_201301291408_2896_r_000000_1: at com.metamx.druid.index.v1.QueryableIndexIndexableAdapter.getDimValueLookup(QueryableIndexIndexableAdapter.java:103)
attempt_201301291408_2896_r_000000_1: at com.metamx.druid.index.v1.IndexMerger.makeIndexFiles(IndexMerger.java:457)
attempt_201301291408_2896_r_000000_1: at com.metamx.druid.index.v1.IndexMerger.merge(IndexMerger.java:294)
attempt_201301291408_2896_r_000000_1: at com.metamx.druid.index.v1.IndexMerger.mergeQueryableIndex(IndexMerger.java:156)
attempt_201301291408_2896_r_000000_1: at com.metamx.druid.indexer.IndexGeneratorJob$IndexGeneratorReducer.reduce(IndexGeneratorJob.java:315)
attempt_201301291408_2896_r_000000_1: at com.metam
@sixtus
sixtus / gist:3053123
Created July 5, 2012 11:24
Compiling sqlite with ICU support
$ ./configure CFLAGS="-Os -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_ICU -DSQLITE_ENABLE_RTREE -DSQLITE_THREADSAFE=2 `icu-config --cppflags`" LDFLAGS="`icu-config --ldflags`" LIBS="-licudata -licui18n -licuio -licule -liculx -licutu -licuuc"
$ make -j4 && make install
$ sqlite3
sqlite> select icu_load_collation('de_DE', 'de_DE');
sqlite> create table t1(name collate de_DE);
sqlite> insert into t1 values('a'), values('ä'), values('o'), values('ö');
sqlite> select * from t1 order by name desc;
ö
o
ä
@sixtus
sixtus / comment.txt
Created May 8, 2012 08:41
Notice that neither can receive any message on sender. It stops receiving messages, when I pass an IP address to bind().
However, I am required to pass an IP address to bind() to make the Mac use it's non-default gw.
So here, I am also showing my work around, which is having two sockets with different configuration.