Skip to content

Instantly share code, notes, and snippets.

@sammcj
Created February 9, 2015 00:04
Show Gist options
  • Save sammcj/66c212311d99a9ca93fe to your computer and use it in GitHub Desktop.
Save sammcj/66c212311d99a9ca93fe to your computer and use it in GitHub Desktop.
logstash_shard_logs
[2015-02-09 11:01:55,754][WARN ][cluster.action.shard ] [int-elastic-01-ES1] [logstash-2015.02.09][4] sending failed shard for [logstash-2015.02.09][4], node[h73_BCI-SW6MkRoE0-3QsA], [R], s[INITIALIZING], indexUUID [ZA9X4b31QJKB3oYlHUhrlA], reason [Failed to start shard, message [RecoveryFailedException[[logstash-2015.02.09][4]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]]; nested: RemoteTransportException[[int-elastic-03-ES1][inet[/10.51.11.189:9300]][internal:index/shard/recovery/start_recovery]]; nested: RecoveryEngineException[[logstash-2015.02.09][4] Phase[2] Execution failed]; nested: RemoteTransportException[[int-elastic-01-ES1][inet[/10.51.11.252:9300]][internal:index/shard/recovery/translog_ops]]; nested: MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [Feb 9 11:00:01], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "Feb 9 11:00:01"]; ]]
[2015-02-09 11:01:55,761][WARN ][cluster.action.shard ] [int-elastic-01-ES1] [logstash-2015.02.09][1] sending failed shard for [logstash-2015.02.09][1], node[h73_BCI-SW6MkRoE0-3QsA], [R], s[INITIALIZING], indexUUID [ZA9X4b31QJKB3oYlHUhrlA], reason [Failed to start shard, message [RecoveryFailedException[[logstash-2015.02.09][1]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]]; nested: RemoteTransportException[[int-elastic-03-ES1][inet[/10.51.11.189:9300]][internal:index/shard/recovery/start_recovery]]; nested: RecoveryEngineException[[logstash-2015.02.09][1] Phase[2] Execution failed]; nested: RemoteTransportException[[int-elastic-01-ES1][inet[/10.51.11.252:9300]][internal:index/shard/recovery/translog_ops]]; nested: MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [Feb 9 11:00:01], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "Feb 9 11:00:01"]; ]]
[2015-02-09 11:01:56,363][WARN ][indices.recovery ] [int-elastic-01-ES1] [logstash-2015.02.09][4] recovery from [[int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]]] failed
Caused by: org.elasticsearch.index.engine.RecoveryEngineException: [logstash-2015.02.09][4] Phase[2] Execution failed
[2015-02-09 11:01:56,363][WARN ][indices.cluster ] [int-elastic-01-ES1] [logstash-2015.02.09][4] failed to start shard
org.elasticsearch.indices.recovery.RecoveryFailedException: [logstash-2015.02.09][4]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]
Caused by: org.elasticsearch.index.engine.RecoveryEngineException: [logstash-2015.02.09][4] Phase[2] Execution failed
[2015-02-09 11:01:56,366][WARN ][indices.recovery ] [int-elastic-01-ES1] [logstash-2015.02.09][1] recovery from [[int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]]] failed
Caused by: org.elasticsearch.index.engine.RecoveryEngineException: [logstash-2015.02.09][1] Phase[2] Execution failed
[2015-02-09 11:01:56,366][WARN ][indices.cluster ] [int-elastic-01-ES1] [logstash-2015.02.09][1] failed to start shard
org.elasticsearch.indices.recovery.RecoveryFailedException: [logstash-2015.02.09][1]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]
Caused by: org.elasticsearch.index.engine.RecoveryEngineException: [logstash-2015.02.09][1] Phase[2] Execution failed
[2015-02-09 11:01:56,372][WARN ][cluster.action.shard ] [int-elastic-01-ES1] [logstash-2015.02.09][4] sending failed shard for [logstash-2015.02.09][4], node[h73_BCI-SW6MkRoE0-3QsA], [R], s[INITIALIZING], indexUUID [ZA9X4b31QJKB3oYlHUhrlA], reason [Failed to start shard, message [RecoveryFailedException[[logstash-2015.02.09][4]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]]; nested: RemoteTransportException[[int-elastic-03-ES1][inet[/10.51.11.189:9300]][internal:index/shard/recovery/start_recovery]]; nested: RecoveryEngineException[[logstash-2015.02.09][4] Phase[2] Execution failed]; nested: RemoteTransportException[[int-elastic-01-ES1][inet[/10.51.11.252:9300]][internal:index/shard/recovery/translog_ops]]; nested: MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [Feb 9 11:00:01], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "Feb 9 11:00:01"]; ]]
[2015-02-09 11:01:56,380][WARN ][cluster.action.shard ] [int-elastic-01-ES1] [logstash-2015.02.09][1] sending failed shard for [logstash-2015.02.09][1], node[h73_BCI-SW6MkRoE0-3QsA], [R], s[INITIALIZING], indexUUID [ZA9X4b31QJKB3oYlHUhrlA], reason [Failed to start shard, message [RecoveryFailedException[[logstash-2015.02.09][1]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]]; nested: RemoteTransportException[[int-elastic-03-ES1][inet[/10.51.11.189:9300]][internal:index/shard/recovery/start_recovery]]; nested: RecoveryEngineException[[logstash-2015.02.09][1] Phase[2] Execution failed]; nested: RemoteTransportException[[int-elastic-01-ES1][inet[/10.51.11.252:9300]][internal:index/shard/recovery/translog_ops]]; nested: MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [Feb 9 11:00:01], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "Feb 9 11:00:01"]; ]]
[2015-02-09 11:01:56,939][WARN ][indices.recovery ] [int-elastic-01-ES1] [logstash-2015.02.09][4] recovery from [[int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]]] failed
Caused by: org.elasticsearch.index.engine.RecoveryEngineException: [logstash-2015.02.09][4] Phase[2] Execution failed
[2015-02-09 11:01:56,939][WARN ][indices.cluster ] [int-elastic-01-ES1] [logstash-2015.02.09][4] failed to start shard
org.elasticsearch.indices.recovery.RecoveryFailedException: [logstash-2015.02.09][4]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]
Caused by: org.elasticsearch.index.engine.RecoveryEngineException: [logstash-2015.02.09][4] Phase[2] Execution failed
[2015-02-09 11:01:56,946][WARN ][cluster.action.shard ] [int-elastic-01-ES1] [logstash-2015.02.09][4] sending failed shard for [logstash-2015.02.09][4], node[h73_BCI-SW6MkRoE0-3QsA], [R], s[INITIALIZING], indexUUID [ZA9X4b31QJKB3oYlHUhrlA], reason [Failed to start shard, message [RecoveryFailedException[[logstash-2015.02.09][4]: Recovery failed from [int-elastic-03-ES1][qlMwQhWoRIedLZ5r23eCcw][int-elastic-03][inet[/10.51.11.189:9300]] into [int-elastic-01-ES1][h73_BCI-SW6MkRoE0-3QsA][int-elastic-01][inet[10.51.11.252/10.51.11.252:9300]]]; nested: RemoteTransportException[[int-elastic-03-ES1][inet[/10.51.11.189:9300]][internal:index/shard/recovery/start_recovery]]; nested: RecoveryEngineException[[logstash-2015.02.09][4] Phase[2] Execution failed]; nested: RemoteTransportException[[int-elastic-01-ES1][inet[/10.51.11.252:9300]][internal:index/shard/recovery/translog_ops]]; nested: MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [Feb 9 11:00:01], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "Feb 9 11:00:01"]; ]]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment