Skip to content

Instantly share code, notes, and snippets.

@jprante
Created November 5, 2013 15:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jprante/7321028 to your computer and use it in GitHub Desktop.
Save jprante/7321028 to your computer and use it in GitHub Desktop.
Elasticsearch 0.90.6 river test
curl -XPUT 'localhost:9200/_river/my_jdbc_river/_meta' -d '{
"type" : "jdbc",
"jdbc" : {
"driver" : "com.mysql.jdbc.Driver",
"url" : "jdbc:mysql://localhost:3306/test",
"user" : "",
"password" : "",
"sql" : "select * from orders",
"strategy" : "simple"
}
}'
{"ok":true,"_index":"_river","_type":"my_jdbc_river","_id":"_meta","_version":1}
[2013-11-05 16:39:18,117][INFO ][node ] [Bora] version[0.90.6], pid[1879], build[e2a24ef/2013-11-04T13:44:16Z]
[2013-11-05 16:39:18,117][INFO ][node ] [Bora] initializing ...
[2013-11-05 16:39:18,118][DEBUG][node ] [Bora] using home [/Users/es/elasticsearch-0.90.6], config [/Users/es/elasticsearch-0.90.6/config], data [[/Users/es/elasticsearch-0.90.6/data]], logs [/Users/es/elasticsearch-0.90.6/logs], work [/Users/es/elasticsearch-0.90.6/work], plugins [/Users/es/elasticsearch-0.90.6/plugins]
[2013-11-05 16:39:18,130][INFO ][plugins ] [Bora] loaded [jdbc-river], sites []
[2013-11-05 16:39:18,162][DEBUG][common.compress.lzf ] using [UnsafeChunkDecoder] decoder
[2013-11-05 16:39:18,178][DEBUG][env ] [Bora] using node location [[/Users/es/elasticsearch-0.90.6/data/elasticsearch/nodes/0]], local_node_id [0]
[2013-11-05 16:39:19,121][DEBUG][threadpool ] [Bora] creating thread_pool [generic], type [cached], keep_alive [30s]
[2013-11-05 16:39:19,128][DEBUG][threadpool ] [Bora] creating thread_pool [index], type [fixed], size [4], queue_size [200]
[2013-11-05 16:39:19,131][DEBUG][threadpool ] [Bora] creating thread_pool [bulk], type [fixed], size [4], queue_size [50]
[2013-11-05 16:39:19,132][DEBUG][threadpool ] [Bora] creating thread_pool [get], type [fixed], size [4], queue_size [1k]
[2013-11-05 16:39:19,132][DEBUG][threadpool ] [Bora] creating thread_pool [search], type [fixed], size [12], queue_size [1k]
[2013-11-05 16:39:19,132][DEBUG][threadpool ] [Bora] creating thread_pool [suggest], type [fixed], size [4], queue_size [1k]
[2013-11-05 16:39:19,132][DEBUG][threadpool ] [Bora] creating thread_pool [percolate], type [fixed], size [4], queue_size [1k]
[2013-11-05 16:39:19,132][DEBUG][threadpool ] [Bora] creating thread_pool [management], type [scaling], min [1], size [5], keep_alive [5m]
[2013-11-05 16:39:19,133][DEBUG][threadpool ] [Bora] creating thread_pool [flush], type [scaling], min [1], size [2], keep_alive [5m]
[2013-11-05 16:39:19,133][DEBUG][threadpool ] [Bora] creating thread_pool [merge], type [scaling], min [1], size [2], keep_alive [5m]
[2013-11-05 16:39:19,134][DEBUG][threadpool ] [Bora] creating thread_pool [refresh], type [scaling], min [1], size [2], keep_alive [5m]
[2013-11-05 16:39:19,134][DEBUG][threadpool ] [Bora] creating thread_pool [warmer], type [scaling], min [1], size [2], keep_alive [5m]
[2013-11-05 16:39:19,134][DEBUG][threadpool ] [Bora] creating thread_pool [snapshot], type [scaling], min [1], size [2], keep_alive [5m]
[2013-11-05 16:39:19,134][DEBUG][threadpool ] [Bora] creating thread_pool [optimize], type [fixed], size [1], queue_size [null]
[2013-11-05 16:39:19,158][DEBUG][transport.netty ] [Bora] using worker_count[8], port[9300-9400], bind_host[null], publish_host[null], compress[false], connect_timeout[30s], connections_per_node[2/3/6/1/1], receive_predictor[512kb->512kb]
[2013-11-05 16:39:19,168][DEBUG][discovery.zen.ping.multicast] [Bora] using group [224.2.2.4], with port [54328], ttl [3], and address [null]
[2013-11-05 16:39:19,171][DEBUG][discovery.zen.ping.unicast] [Bora] using initial hosts [], with concurrent_connects [10]
[2013-11-05 16:39:19,172][DEBUG][discovery.zen ] [Bora] using ping.timeout [3s], master_election.filter_client [true], master_election.filter_data [false]
[2013-11-05 16:39:19,172][DEBUG][discovery.zen.elect ] [Bora] using minimum_master_nodes [-1]
[2013-11-05 16:39:19,173][DEBUG][discovery.zen.fd ] [Bora] [master] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2013-11-05 16:39:19,176][DEBUG][discovery.zen.fd ] [Bora] [node ] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2013-11-05 16:39:19,202][DEBUG][monitor.jvm ] [Bora] enabled [true], last_gc_enabled [false], interval [1s], gc_threshold [{default=GcThreshold{name='default', warnThreshold=10000, infoThreshold=5000, debugThreshold=2000}, ParNew=GcThreshold{name='ParNew', warnThreshold=1000, infoThreshold=700, debugThreshold=400}, ConcurrentMarkSweep=GcThreshold{name='ConcurrentMarkSweep', warnThreshold=10000, infoThreshold=5000, debugThreshold=2000}}]
[2013-11-05 16:39:19,712][DEBUG][monitor.os ] [Bora] Using probe [org.elasticsearch.monitor.os.SigarOsProbe@273f212a] with refresh_interval [1s]
[2013-11-05 16:39:19,718][DEBUG][monitor.process ] [Bora] Using probe [org.elasticsearch.monitor.process.SigarProcessProbe@77b9e7fc] with refresh_interval [1s]
[2013-11-05 16:39:19,724][DEBUG][monitor.jvm ] [Bora] Using refresh_interval [1s]
[2013-11-05 16:39:19,725][DEBUG][monitor.network ] [Bora] Using probe [org.elasticsearch.monitor.network.SigarNetworkProbe@53b258fa] with refresh_interval [5s]
[2013-11-05 16:39:19,735][DEBUG][monitor.network ] [Bora] net_info
host [Jorg-Prantes-MacBook-Pro.local]
en0 display_name [en0]
address [/fe80:0:0:0:129a:ddff:fe52:72e8%4] [/10.1.1.79]
mtu [1500] multicast [true] ptp [false] loopback [false] up [true] virtual [false]
lo0 display_name [lo0]
address [/0:0:0:0:0:0:0:1] [/fe80:0:0:0:0:0:0:1%1] [/127.0.0.1]
mtu [16384] multicast [true] ptp [false] loopback [true] up [true] virtual [false]
[2013-11-05 16:39:19,739][DEBUG][monitor.fs ] [Bora] Using probe [org.elasticsearch.monitor.fs.SigarFsProbe@6e2c5444] with refresh_interval [1s]
[2013-11-05 16:39:19,986][DEBUG][indices.store ] [Bora] using indices.store.throttle.type [MERGE], with index.store.throttle.max_bytes_per_sec [20mb]
[2013-11-05 16:39:19,992][DEBUG][cache.memory ] [Bora] using bytebuffer cache with small_buffer_size [1kb], large_buffer_size [1mb], small_cache_size [10mb], large_cache_size [500mb], direct [true]
[2013-11-05 16:39:20,001][DEBUG][script ] [Bora] using script cache with max_size [500], expire [null]
[2013-11-05 16:39:20,007][DEBUG][cluster.routing.allocation.decider] [Bora] using node_concurrent_recoveries [2], node_initial_primaries_recoveries [4]
[2013-11-05 16:39:20,008][DEBUG][cluster.routing.allocation.decider] [Bora] using [cluster.routing.allocation.allow_rebalance] with [indices_all_active]
[2013-11-05 16:39:20,008][DEBUG][cluster.routing.allocation.decider] [Bora] using [cluster_concurrent_rebalance] with [2]
[2013-11-05 16:39:20,012][DEBUG][gateway.local ] [Bora] using initial_shards [quorum], list_timeout [30s]
[2013-11-05 16:39:20,115][DEBUG][indices.recovery ] [Bora] using max_bytes_per_sec[20mb], concurrent_streams [3], file_chunk_size [512kb], translog_size [512kb], translog_ops [1000], and compress [true]
[2013-11-05 16:39:20,163][DEBUG][http.netty ] [Bora] using max_chunk_size[8kb], max_header_size[8kb], max_initial_line_length[4kb], max_content_length[100mb], receive_predictor[512kb->512kb]
[2013-11-05 16:39:20,166][DEBUG][indices.memory ] [Bora] using index_buffer_size [101.5mb], with min_shard_index_buffer_size [4mb], max_shard_index_buffer_size [512mb], shard_inactive_time [30m]
[2013-11-05 16:39:20,168][DEBUG][indices.cache.filter ] [Bora] using [node] weighted filter cache with size [20%], actual_size [203.1mb], expire [null], clean_interval [1m]
[2013-11-05 16:39:20,169][DEBUG][indices.fielddata.cache ] [Bora] using size [-1] [-1b], expire [null]
[2013-11-05 16:39:20,184][DEBUG][gateway.local.state.meta ] [Bora] using gateway.local.auto_import_dangled [YES], with gateway.local.dangling_timeout [2h]
[2013-11-05 16:39:20,200][DEBUG][gateway.local.state.meta ] [Bora] took 16ms to load state
[2013-11-05 16:39:20,211][DEBUG][gateway.local.state.shards] [Bora] took 9ms to load started shards state
[2013-11-05 16:39:20,214][DEBUG][bulk.udp ] [Bora] using enabled [false], host [null], port [9700-9800], bulk_actions [1000], bulk_size [5mb], flush_interval [5s], concurrent_requests [4]
[2013-11-05 16:39:20,218][DEBUG][cluster.routing.allocation.decider] [Bora] using node_concurrent_recoveries [2], node_initial_primaries_recoveries [4]
[2013-11-05 16:39:20,219][DEBUG][cluster.routing.allocation.decider] [Bora] using [cluster.routing.allocation.allow_rebalance] with [indices_all_active]
[2013-11-05 16:39:20,219][DEBUG][cluster.routing.allocation.decider] [Bora] using [cluster_concurrent_rebalance] with [2]
[2013-11-05 16:39:20,220][DEBUG][cluster.routing.allocation.decider] [Bora] using node_concurrent_recoveries [2], node_initial_primaries_recoveries [4]
[2013-11-05 16:39:20,220][DEBUG][cluster.routing.allocation.decider] [Bora] using [cluster.routing.allocation.allow_rebalance] with [indices_all_active]
[2013-11-05 16:39:20,220][DEBUG][cluster.routing.allocation.decider] [Bora] using [cluster_concurrent_rebalance] with [2]
[2013-11-05 16:39:20,231][INFO ][node ] [Bora] initialized
[2013-11-05 16:39:20,231][INFO ][node ] [Bora] starting ...
[2013-11-05 16:39:20,245][DEBUG][netty.channel.socket.nio.SelectorUtil] Using select timeout of 500
[2013-11-05 16:39:20,245][DEBUG][netty.channel.socket.nio.SelectorUtil] Epoll-bug workaround enabled = false
[2013-11-05 16:39:20,324][DEBUG][transport.netty ] [Bora] Bound to address [/0:0:0:0:0:0:0:0%0:9300]
[2013-11-05 16:39:20,327][INFO ][transport ] [Bora] bound_address {inet[/0:0:0:0:0:0:0:0%0:9300]}, publish_address {inet[/10.1.1.79:9300]}
[2013-11-05 16:39:23,383][DEBUG][discovery.zen ] [Bora] filtered ping responses: (filter_client[true], filter_data[false]) {none}
[2013-11-05 16:39:23,387][DEBUG][cluster.service ] [Bora] processing [zen-disco-join (elected_as_master)]: execute
[2013-11-05 16:39:23,388][DEBUG][cluster.service ] [Bora] cluster state updated, version [1], source [zen-disco-join (elected_as_master)]
[2013-11-05 16:39:23,389][INFO ][cluster.service ] [Bora] new_master [Bora][wvk1bbV4SXyIG36DkHjh8g][inet[/10.1.1.79:9300]], reason: zen-disco-join (elected_as_master)
[2013-11-05 16:39:23,425][DEBUG][transport.netty ] [Bora] connected to node [[Bora][wvk1bbV4SXyIG36DkHjh8g][inet[/10.1.1.79:9300]]]
[2013-11-05 16:39:23,426][DEBUG][cluster.service ] [Bora] publishing cluster state version 1
[2013-11-05 16:39:23,426][DEBUG][cluster.service ] [Bora] set local cluster state to version 1
[2013-11-05 16:39:23,428][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: execute
[2013-11-05 16:39:23,428][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: no change in cluster_state
[2013-11-05 16:39:23,429][DEBUG][cluster.service ] [Bora] processing [zen-disco-join (elected_as_master)]: done applying updated cluster_state (version: 1)
[2013-11-05 16:39:23,429][INFO ][discovery ] [Bora] elasticsearch/wvk1bbV4SXyIG36DkHjh8g
[2013-11-05 16:39:23,440][DEBUG][cluster.service ] [Bora] processing [local-gateway-elected-state]: execute
[2013-11-05 16:39:23,445][INFO ][http ] [Bora] bound_address {inet[/0:0:0:0:0:0:0:0%0:9200]}, publish_address {inet[/10.1.1.79:9200]}
[2013-11-05 16:39:23,446][INFO ][node ] [Bora] started
[2013-11-05 16:39:23,450][DEBUG][gateway.local ] [Bora] [_river][0]: allocating [[_river][0], node[null], [P], s[UNASSIGNED]] to [[Bora][wvk1bbV4SXyIG36DkHjh8g][inet[/10.1.1.79:9300]]] on primary allocation
[2013-11-05 16:39:23,456][DEBUG][cluster.service ] [Bora] cluster state updated, version [2], source [local-gateway-elected-state]
[2013-11-05 16:39:23,456][DEBUG][cluster.service ] [Bora] publishing cluster state version 2
[2013-11-05 16:39:23,457][DEBUG][cluster.service ] [Bora] set local cluster state to version 2
[2013-11-05 16:39:23,458][DEBUG][indices.cluster ] [Bora] [_river] creating index
[2013-11-05 16:39:23,458][DEBUG][indices ] [Bora] creating Index [_river], shards [1]/[1]
[2013-11-05 16:39:23,685][DEBUG][index.mapper ] [Bora] [_river] using dynamic[true], default mapping: default_mapping_location[null], loaded_from[jar:file:/Users/es/elasticsearch-0.90.6/lib/elasticsearch-0.90.6.jar!/org/elasticsearch/index/mapper/default-mapping.json]
[2013-11-05 16:39:23,686][DEBUG][index.cache.query.parser.resident] [Bora] [_river] using [resident] query cache with max_size [100], expire [null]
[2013-11-05 16:39:23,694][DEBUG][index.store.fs ] [Bora] [_river] using index.store.throttle.type [node], with index.store.throttle.max_bytes_per_sec [0b]
[2013-11-05 16:39:23,746][DEBUG][indices.cluster ] [Bora] [_river][0] creating shard
[2013-11-05 16:39:23,746][DEBUG][index.service ] [Bora] [_river] creating shard_id [0]
[2013-11-05 16:39:23,829][DEBUG][index.deletionpolicy ] [Bora] [_river][0] Using [keep_only_last] deletion policy
[2013-11-05 16:39:23,830][DEBUG][index.merge.policy ] [Bora] [_river][0] using [tiered] merge policy with expunge_deletes_allowed[10.0], floor_segment[2mb], max_merge_at_once[10], max_merge_at_once_explicit[30], max_merged_segment[5gb], segments_per_tier[10.0], reclaim_deletes_weight[2.0], async_merge[true]
[2013-11-05 16:39:23,831][DEBUG][index.merge.scheduler ] [Bora] [_river][0] using [concurrent] merge scheduler with max_thread_count[2]
[2013-11-05 16:39:23,835][DEBUG][index.shard.service ] [Bora] [_river][0] state: [CREATED]
[2013-11-05 16:39:23,835][DEBUG][index.translog ] [Bora] [_river][0] interval [5s], flush_threshold_ops [5000], flush_threshold_size [200mb], flush_threshold_period [30m]
[2013-11-05 16:39:23,838][DEBUG][index.shard.service ] [Bora] [_river][0] state: [CREATED]->[RECOVERING], reason [from gateway]
[2013-11-05 16:39:23,839][DEBUG][index.gateway ] [Bora] [_river][0] starting recovery from local ...
[2013-11-05 16:39:23,843][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: execute
[2013-11-05 16:39:23,844][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: no change in cluster_state
[2013-11-05 16:39:23,850][DEBUG][index.engine.robin ] [Bora] [_river][0] starting engine
[2013-11-05 16:39:23,853][INFO ][gateway ] [Bora] recovered [1] indices into cluster_state
[2013-11-05 16:39:23,853][DEBUG][cluster.service ] [Bora] processing [local-gateway-elected-state]: done applying updated cluster_state (version: 2)
[2013-11-05 16:39:23,972][DEBUG][index.shard.service ] [Bora] [_river][0] state: [RECOVERING]->[POST_RECOVERY], reason [post recovery]
[2013-11-05 16:39:23,973][DEBUG][index.shard.service ] [Bora] [_river][0] scheduling refresher every 1s
[2013-11-05 16:39:23,973][DEBUG][index.shard.service ] [Bora] [_river][0] scheduling optimizer / merger every 1s
[2013-11-05 16:39:23,973][DEBUG][index.gateway ] [Bora] [_river][0] recovery completed from [local], took [134ms]
[2013-11-05 16:39:23,974][DEBUG][cluster.action.shard ] [Bora] sending shard started for [_river][0], node[wvk1bbV4SXyIG36DkHjh8g], [P], s[INITIALIZING], indexUUID [Ybi3q7xXRLK4GJrg279KYg], reason [after recovery from gateway]
[2013-11-05 16:39:23,974][DEBUG][cluster.action.shard ] [Bora] received shard started for [_river][0], node[wvk1bbV4SXyIG36DkHjh8g], [P], s[INITIALIZING], indexUUID [Ybi3q7xXRLK4GJrg279KYg], reason [after recovery from gateway]
[2013-11-05 16:39:23,974][DEBUG][cluster.service ] [Bora] processing [shard-started ([_river][0], node[wvk1bbV4SXyIG36DkHjh8g], [P], s[INITIALIZING]), reason [after recovery from gateway]]: execute
[2013-11-05 16:39:23,975][DEBUG][cluster.action.shard ] [Bora] [_river][0] will apply shard started [_river][0], node[wvk1bbV4SXyIG36DkHjh8g], [P], s[INITIALIZING], indexUUID [Ybi3q7xXRLK4GJrg279KYg], reason [after recovery from gateway]
[2013-11-05 16:39:23,975][DEBUG][cluster.service ] [Bora] cluster state updated, version [3], source [shard-started ([_river][0], node[wvk1bbV4SXyIG36DkHjh8g], [P], s[INITIALIZING]), reason [after recovery from gateway]]
[2013-11-05 16:39:23,975][DEBUG][cluster.service ] [Bora] publishing cluster state version 3
[2013-11-05 16:39:23,976][DEBUG][cluster.service ] [Bora] set local cluster state to version 3
[2013-11-05 16:39:23,976][DEBUG][index.shard.service ] [Bora] [_river][0] state: [POST_RECOVERY]->[STARTED], reason [global state is [STARTED]]
[2013-11-05 16:39:23,976][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: execute
[2013-11-05 16:39:23,976][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: no change in cluster_state
[2013-11-05 16:39:23,987][DEBUG][cluster.service ] [Bora] processing [shard-started ([_river][0], node[wvk1bbV4SXyIG36DkHjh8g], [P], s[INITIALIZING]), reason [after recovery from gateway]]: done applying updated cluster_state (version: 3)
[2013-11-05 16:39:33,428][DEBUG][cluster.service ] [Bora] processing [routing-table-updater]: execute
[2013-11-05 16:39:33,429][DEBUG][cluster.service ] [Bora] processing [routing-table-updater]: no change in cluster_state
[2013-11-05 16:39:46,912][DEBUG][cluster.service ] [Bora] processing [update-mapping [_river][my_jdbc_river]]: execute
[2013-11-05 16:39:46,919][DEBUG][cluster.metadata ] [Bora] [_river] update_mapping [my_jdbc_river] (dynamic) with source [{"my_jdbc_river":{"properties":{"jdbc":{"properties":{"driver":{"type":"string"},"sql":{"type":"string"},"strategy":{"type":"string"},"url":{"type":"string"}}},"type":{"type":"string"}}}}]
[2013-11-05 16:39:46,925][DEBUG][cluster.service ] [Bora] cluster state updated, version [4], source [update-mapping [_river][my_jdbc_river]]
[2013-11-05 16:39:46,925][DEBUG][cluster.service ] [Bora] publishing cluster state version 4
[2013-11-05 16:39:46,926][DEBUG][cluster.service ] [Bora] set local cluster state to version 4
[2013-11-05 16:39:46,926][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: execute
[2013-11-05 16:39:46,931][DEBUG][cluster.service ] [Bora] processing [update-mapping [_river][my_jdbc_river]]: done applying updated cluster_state (version: 4)
[2013-11-05 16:39:46,932][DEBUG][river.cluster ] [Bora] processing [reroute_rivers_node_changed]: no change in cluster_state
[2013-11-05 16:39:50,233][DEBUG][indices.memory ] [Bora] recalculating shard indexing buffer (reason=active/inactive[false] created/deleted[true]), total is [101.5mb] with [1] active shards, each shard set to indexing=[101.5mb], translog=[64kb]
[2013-11-05 16:39:50,233][DEBUG][index.engine.robin ] [Bora] [_river][0] updating index_buffer_size from [64mb] to [101.5mb]
@xubingnuo
Copy link

which firend can help?i have same problems too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment