Skip to content

Instantly share code, notes, and snippets.

Peter karussell

Block or report user

Report or block karussell

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@karussell
karussell / doc.md
Last active Aug 29, 2015
mapsforge map creation
View doc.md

You'll need /data and /install folders, if this is not desired you'll need to change the script variables for the paths

  1. install osmosis 0.43.1 http://bretth.dev.openstreetmap.org/osmosis-build/ to /install/osmosis
  2. do git clone https://github.com/graphhopper/graphhopper /install/graphhopper_mapscreation then do git checkout 0.3
  3. try cd /install/graphhopper_mapscreation; ./graphhopper.sh web europe_germany_berlin.pbf, this should not fail
  4. create a new directory which should be accessible from outside /data/public/maps/ and contains subdirectory '0.4'
  5. create /data/osm and /data/mapsforge where pbf and map files will be stored
  6. install java to /install/java so that one can execute /install/java/bin/java
  7. copy mapsforge writer 0.4.3 to /home//.openstreetmap/osmosis/plugins/mapsforge-map-writer-0.4.3-jar-with-dependencies.jar from http://ci.mapsforge.org/job/release-0.4.3/ or from maven central
  8. include as many map names for de_maps, us_maps, eu_maps as possible. The naming
View RAMCacheTest.java
// dependency: graphhopper 0.3
// JVM_OPTS="-Xms2400m -Xmx2400m"
//
// result on my oldish machine:
// separate arrays: GET=34.10 and SET=36.34
// single array: GET=36.34 and SET=34.49
//
// => GET is ~7% faster for single array
// => SET is ~5% faster for single array
//
View JetwickFilterFactory.java
package de.jetwick.es;
import org.apache.lucene.analysis.CharArraySet;
import org.apache.lucene.analysis.TokenStream;
import de.jetwick.org.apache.solr.analysis.WordDelimiterFilter;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.index.Index;
import org.elasticsearch.index.analysis.AbstractTokenFilterFactory;
public class JetwickFilterFactory extends AbstractTokenFilterFactory {
@karussell
karussell / es-exception-for-custom-filter
Created Jan 20, 2011
elastic search exception for custom filter
View es-exception-for-custom-filter
2011-01-21 00:20:48,468 [elasticsearch[Smuggler II]clusterService#updateTask-pool-6-thread-1] ERROR org.elasticsearch.gateway - [Smuggler II] failed to create index [twindexreal]
org.elasticsearch.common.inject.CreationException: Guice creation errors:
1) No implementation for java.util.Map<java.lang.String, org.elasticsearch.index.analysis.AnalyzerProviderFactory> was bound.
while locating java.util.Map<java.lang.String, org.elasticsearch.index.analysis.AnalyzerProviderFactory>
for parameter 2 at org.elasticsearch.index.analysis.AnalysisService.<init>(AnalysisService.java:59)
while locating org.elasticsearch.index.analysis.AnalysisService
for parameter 3 at org.elasticsearch.index.mapper.MapperService.<init>(MapperService.java:89)
at org.elasticsearch.index.mapper.MapperServiceModule.configure(MapperServiceModule.java:30)
View gist:816297
for (String fromIndex : indexList) {
SearchResponse rsp = client.prepareSearch(fromIndex).
setQuery(QueryBuilders.matchAllQuery()).setSize(hitsPerPage).
// This is important:
setSearchType(SearchType.QUERY_AND_FETCH).
setScroll(TimeValue.timeValueMinutes(30)).execute().actionGet();
try {
long total = rsp.hits().totalHits();
Collection<MyTweet> tweets = collectTweets(rsp);
bulkUpdate(tweets, intoIndex);
@karussell
karussell / TestES.java
Created Feb 8, 2011
Something goes wrong when doing bulkUpdate and retrieving
View TestES.java
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import java.util.Collection;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchType;
import org.elasticsearch.common.unit.TimeValue;
import java.util.LinkedHashMap;
import java.util.Map.Entry;
View exception on client
org.elasticsearch.transport.RemoteTransportException: [Fer-de-Lance][inet[/127.0.0.1:9300]][indices/search]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], total failure; shardFailures {[VHXZa25OQGCtSDa9zdUyFQ][twindex][0]: QueryPhaseExecutionException[[twindex][0]: query[ConstantScore(org.elasticsearch.common.lucene.search.AndFilter@7ed467ba)],from[0],size[15],sort[<custom:"relevance": org.elasticsearch.index.field.data.doubles.DoubleFieldDataType$1@1afc83a>!]: Query Failed [Failed to execute main query]]; nested: }{[VHXZa25OQGCtSDa9zdUyFQ][twindex][3]: QueryPhaseExecutionException[[twindex][3]: query[ConstantScore(org.elasticsearch.common.lucene.search.AndFilter@7ed467ba)],from[0],size[15],sort[<custom:"relevance": org.elasticsearch.index.field.data.doubles.DoubleFieldDataType$1@1afc83a>!]: Query Failed [Failed to execute main query]]; nested: }
...
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhas
@karussell
karussell / search.java
Created Feb 12, 2011
search twitter via maxId
View search.java
// we don't want results with an id larger than maxId
long maxId = lastId;
// we don't want results with an id smaller than sinceId
long sinceId = lastId;
int hitsPerPage = 100;
int maxPages = 1;
boolean breakPaging = false;
for (int page = 0; page < maxPages; page++) {
Query query = new Query(term);
@karussell
karussell / result.txt
Created Feb 22, 2011
different facet entries but only sort changed
View result.txt
a document with a number -> faceting over that
ComparatorType.TOTAL
====================
term:9 count:2 total:2.0
there should be 10 terms. there were:1
ComparatorType.COUNT
====================
View TermsStatsFacetsShouldAllowFilter.java
@Test public void testTermsStatsFacetsShouldAllowFilter() throws Exception {
try {
client.admin().indices().prepareDelete("test").execute().actionGet();
} catch (Exception e) {
// ignore
}
client.admin().indices().prepareCreate("test").execute().actionGet();
client.admin().cluster().prepareHealth().setWaitForGreenStatus().execute().actionGet();
for (int i = 0; i < 20; i++) {
You can’t perform that action at this time.