dblessing

grok {
  match => [
             'message', '(?m)%{DATESTAMP:timestamp}  ?%{LOGLEVEL:loglevel} %{DATA:category}:%{INT:line} - %{GREEDYDATA:message}',
             'message', '(?m)%{GREEDYDATA:timestamp} %{JAVACLASS:class} %{WORD:method} %{GREEDYDATA:message}',
             'message', '(?m)%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:loglevel} \[%{JAVACLASS:class}\] - %{GREEDYDATA:message}'
           ]
  overwrite => 'message'
}

dblessing

ebug: Augeas[rsyslog preserve fqdn](provider=augeas): /augeas/files/etc/rsyslog.conf/error/path = /files/etc/rsyslog.conf
Debug: Augeas[rsyslog preserve fqdn](provider=augeas): /augeas/files/etc/rsyslog.conf/error/lens = /usr/share/augeas/lenses/dist/rsyslog.aug:30.10-.57:
Debug: Augeas[rsyslog preserve fqdn](provider=augeas): /augeas/files/etc/rsyslog.conf/error/message = Failed to match
    ({ } | { /#comment/ = /[^\001-\004\t\n !+-][^\001-\004\n]*[^\001-\004\t\n ]|[^\001-\004\t\n !+-]/ } | { /entry/ } | { /\\$[0-9A-Za-z]+/ = /[^\001-\004\t\n #,]+/ })*({ /program/ } | { /hostname/ })*
  with tree
    { "$PreserveFQDN" } { "#comment" = "rsyslog v5 configuration file" } {  } { "#comment" = "For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html" } { "#comment" = "If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html" } {  } { "#comment" = "$PreserveFQDN on" } {  } { "#comment" = "### MODULES ####" } {  } { "$ModLoad" = "imuxsock" } { "$ModLoad" = "imklog" } { "#comment" = "

dblessing

Puppet::Type.type(:etcd_key).provide(:etcd) do
  confine feature: :etcd

  def exists?
    debug('returning false')
    false
  end

  def create
    fail 'create'

dblessing

Puppet::Type.type(:etcd_key).provide(:etcd) do
  confine feature: :etcd

  def exists?
    debug('returning false')
    false
  end

  def create
    fail 'create'

dblessing

We have been struggling with this issue for a few months. We've experienced it in versions 0.90.6 - 0.90.13 and now in 
1.1, too. 

We get this error during or after our data loader indexes data. Sometimes it takes a day or two to occur but most recently 
it's been immediately on/after first index. The shard that fails is always in the same index. This is a 2-node cluster 
running on CentOS 6.5 with Oracle Java 1.7.0u51. In addition, there is 1 non-data node for the process that handles 
indexing data and 2 non-data nodes serving the front-end. All non-data nodes are java clients using spring-data-elasticsearch
library. All nodes are on 1.1 now. We understand there's a probability that our loader application is causing this but we 
can't see how or where. Also, it seems like a bug if a client can cause shards to fail on the server. We are grasping at 
straws now and appreciate any ideas on what could be causing this. Stack below. Thanks for your help.

dblessing

This is the other node in the cluster. It doesn't have a failed shard but does give this message at the same 
time the shard fails on the other node.
------

[2014-04-24 14:04:06,732][WARN ][cluster.action.shard     ] [servicebus-es1.bkeprd.com] [product][0] 
received shard failed for [product][0], node[ImXrGRu0SXm0rgQWKEu-kw], [P], s[STARTED], 
indexUUID [SIOpzgF_RoWF6MBGnx4N8w], reason [engine failure, message [MergeException[org.apache.lucene.store.
AlreadyClosedException: this IndexReader is closed]; nested: AlreadyClosedException[this IndexReader is closed]; ]]

dblessing

LoadError: no such file to load -- i18n
  require at org/jruby/RubyKernel.java:1085
  require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
  require at /opt/logstash/lib/logstash/JRUBY-6970.rb:27
   (root) at /opt/logstash/lib/logstash/runner.rb:50

dblessing

tail: /opt/logstash/.sincedb: file truncated
141968 0 64768 446345704
918719 0 64768 888661
144151 0 64768 2661593
## RESTART HERE
tail: /opt/logstash/.sincedb: file truncated
141968 0 64768 446345704
918719 0 64768 888661
144151 0 64768 2661593
## RESTART HERE

dblessing

body {
   font-size: 50px !important;
}

dblessing

body, p, table, tr, td, strong, h1, h2, h3, h4, h5, h6, blockquote, li {
   font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
}

a, a:visited, a:active, a:hover, a:link {
   text-decoration: none;
}

#top-panel {
   background-image: none;