Experimental CLI interface for the helpers in the `python library`_.
Main purpose is to expose the bulk functionality to enable rapid loading of data into an elasticsearch cluster. Combined with the scan command it can also be used to reindex data from elasticsearch into a different index or cluster.
Name the repository whatever you like. In this case, testrepository.
PUT /_snapshot/testrepository
| #!/bin/sh | |
| unset CDPATH | |
| . "$(cd `dirname $0`/..; pwd)/bin/logstash.lib.sh" | |
| setup | |
| for file in /etc/default/logstash /etc/sysconfig/logstash; do | |
| if [ -f "$file" ]; then | |
| . $file | |
| fi |
| buh@Elasticbox (03:18 PM) ~/git/logstash | |
| » rm -rf vendor | |
| buh@Elasticbox (03:18 PM) ~/git/logstash | |
| » rake bootstrap | |
| mkdir -p vendor | |
| mkdir vendor/_ | |
| Downloading http://jruby.org.s3.amazonaws.com/downloads/1.7.25/jruby-bin-1.7.25.tar.gz | |
| Installing minitar >= 0 because the build process needs it. | |
| [bootstrap] Fetching and installing gem: minitar (>= 0) | |
| Fetching: minitar-0.5.4.gem (100%) |
| 2016/01/06 23:15:27.708044 publish.go:100: DBG Publish: { | |
| "@timestamp": "2016-01-06T23:15:23.210Z", | |
| "beat": { | |
| "hostname": "ip-172-31-46-141", | |
| "name": "ip-172-31-46-141" | |
| }, | |
| "count": 1, | |
| "fields": null, | |
| "input_type": "log", | |
| "message": "{ \"@timestamp\": \"2016-01-06T23:15:20+00:00\", \"@version\": \"1\", \"vhost\": \"untergeek.com\", \"clientip\": \"80.240.139.191\", \"bytes\": 60375, \"duration\": 0.114, \"status\": 200, \"request\": \"/feed/\", \"method\": \"GET\", \"referrer\": \"-\", \"useragent\": \"Fever/1.39 (Feed Parser; http://feedafever.com; Allow like Gecko)\" }", |
| failures in bulk class expected behavior | |
| should retry an event until max_retries reached | |
| RSpec::Mocks::MockExpectationError: (LogStash::Outputs::ElasticSearch: {"manage_template"=>true, "index"=>"logstash-2014.11.17", "template_overwrite"=>true, "hosts"=>["127.0.0.1"], "retry_max_items"=>10, "retry_max_interval"=>1, "max_retries"=>3, "codec"=><LogStash::Codecs::Plain charset=>"UTF-8">, "workers"=>1, "template_name"=>"logstash", "flush_size"=>500, "idle_flush_time"=>1, "action"=>"index", "path"=>"/", "ssl"=>false, "ssl_certificate_verification"=>true, "sniffing"=>false, "sniffing_delay"=>5, "doc_as_upsert"=>false, "upsert"=>""}).submit([["index", {:_id=>nil, :_index=>"logstash-2014.11.17", :_type=>"logs", :_routing=>nil}, #<LogStash::Event:0x6ca5d544 @metadata_accessors=#<LogStash::Util::Accessors:0x360bb9b1 @store={"retry_count"=>3}, @lut={}>, @cancelled=false, @data={"somevalue"=>100, "@timestamp"=>"2014-11-17T20:37:17.223Z", "@version"=>"1"}, @metadata={"retry_count"=>3}, @accessors=#<LogStash::Util::Acce |
| input { | |
| heartbeat { | |
| message => "epoch" | |
| interval => 10 | |
| add_field => { "zabbix_host" => "host.example.tld" "zabbix_key" => "ls_heartbeat" } | |
| tags => [ "heartbeat" ] | |
| } | |
| } | |
| filter { } |
| { | |
| "template" : "logstash-*", | |
| "settings" : { | |
| "index.refresh_interval" : "5s" | |
| }, | |
| "mappings" : { | |
| "_default_" : { | |
| "_all" : {"enabled" : true, "omit_norms" : true}, | |
| "dynamic_templates" : [ { | |
| "message_field" : { |
| input { | |
| # file { | |
| # path => ["C:/logs/Proj/*/*.log"] | |
| # start_position => beginning | |
| # } | |
| stdin { } | |
| } |
Please install in this order.