Skip to content

Instantly share code, notes, and snippets.

karussell /
Last active August 29, 2015 14:01
mapsforge map creation

You'll need /data and /install folders, if this is not desired you'll need to change the script variables for the paths

  1. install osmosis 0.43.1 to /install/osmosis
  2. do git clone /install/graphhopper_mapscreation then do git checkout 0.3
  3. try cd /install/graphhopper_mapscreation; ./ web europe_germany_berlin.pbf, this should not fail
  4. create a new directory which should be accessible from outside /data/public/maps/ and contains subdirectory '0.4'
  5. create /data/osm and /data/mapsforge where pbf and map files will be stored
  6. install java to /install/java so that one can execute /install/java/bin/java
  7. copy mapsforge writer 0.4.3 to /home//.openstreetmap/osmosis/plugins/mapsforge-map-writer-0.4.3-jar-with-dependencies.jar from or from maven central
  8. include as many map names for de_maps, us_maps, eu_maps as possible. The naming
karussell /
Last active August 29, 2015 14:14
RAM Cache Test
// dependency: graphhopper 0.3
// JVM_OPTS="-Xms2400m -Xmx2400m"
// result on my oldish machine:
// separate arrays: GET=34.10 and SET=36.34
// single array: GET=36.34 and SET=34.49
// => GET is ~7% faster for single array
// => SET is ~5% faster for single array
import org.apache.lucene.analysis.CharArraySet;
import org.apache.lucene.analysis.TokenStream;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.index.Index;
import org.elasticsearch.index.analysis.AbstractTokenFilterFactory;
public class JetwickFilterFactory extends AbstractTokenFilterFactory {
karussell / es-exception-for-custom-filter
Created January 20, 2011 23:31
elastic search exception for custom filter
2011-01-21 00:20:48,468 [elasticsearch[Smuggler II]clusterService#updateTask-pool-6-thread-1] ERROR org.elasticsearch.gateway - [Smuggler II] failed to create index [twindexreal]
org.elasticsearch.common.inject.CreationException: Guice creation errors:
1) No implementation for java.util.Map<java.lang.String, org.elasticsearch.index.analysis.AnalyzerProviderFactory> was bound.
while locating java.util.Map<java.lang.String, org.elasticsearch.index.analysis.AnalyzerProviderFactory>
for parameter 2 at org.elasticsearch.index.analysis.AnalysisService.<init>(
while locating org.elasticsearch.index.analysis.AnalysisService
for parameter 3 at org.elasticsearch.index.mapper.MapperService.<init>(
at org.elasticsearch.index.mapper.MapperServiceModule.configure(
karussell / gist:816297
Created February 8, 2011 11:34
merge indices
for (String fromIndex : indexList) {
SearchResponse rsp = client.prepareSearch(fromIndex).
// This is important:
try {
long total = rsp.hits().totalHits();
Collection<MyTweet> tweets = collectTweets(rsp);
bulkUpdate(tweets, intoIndex);
karussell /
Created February 8, 2011 20:02
Something goes wrong when doing bulkUpdate and retrieving
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Collection;
import org.elasticsearch.common.unit.TimeValue;
import java.util.LinkedHashMap;
import java.util.Map.Entry;
karussell / exception on client
Created February 10, 2011 22:11
setFilter exception
org.elasticsearch.transport.RemoteTransportException: [Fer-de-Lance][inet[/]][indices/search]
Caused by: Failed to execute phase [query], total failure; shardFailures {[VHXZa25OQGCtSDa9zdUyFQ][twindex][0]: QueryPhaseExecutionException[[twindex][0]: query[ConstantScore(],from[0],size[15],sort[<custom:"relevance":$1@1afc83a>!]: Query Failed [Failed to execute main query]]; nested: }{[VHXZa25OQGCtSDa9zdUyFQ][twindex][3]: QueryPhaseExecutionException[[twindex][3]: query[ConstantScore(],from[0],size[15],sort[<custom:"relevance":$1@1afc83a>!]: Query Failed [Failed to execute main query]]; nested: }
karussell /
Created February 12, 2011 19:09
search twitter via maxId
// we don't want results with an id larger than maxId
long maxId = lastId;
// we don't want results with an id smaller than sinceId
long sinceId = lastId;
int hitsPerPage = 100;
int maxPages = 1;
boolean breakPaging = false;
for (int page = 0; page < maxPages; page++) {
Query query = new Query(term);
karussell / result.txt
Created February 22, 2011 21:33
different facet entries but only sort changed
a document with a number -> faceting over that
term:9 count:2 total:2.0
there should be 10 terms. there were:1
@Test public void testTermsStatsFacetsShouldAllowFilter() throws Exception {
try {
} catch (Exception e) {
// ignore
for (int i = 0; i < 20; i++) {