Skip to content

Instantly share code, notes, and snippets.

@darkone23
Last active December 27, 2015 05:19
Show Gist options
  • Save darkone23/7273745 to your computer and use it in GitHub Desktop.
Save darkone23/7273745 to your computer and use it in GitHub Desktop.
does this look like it will match the way I am expecting?
31 Oct 2013 11:45:08,436 WARN [lifecycleSupervisor-1-2] (org.apache.flume.channel.file.Log.replay:442) - Checkpoint may not have completed successfully. Forcing full replay, this may take a while.
org.apache.flume.channel.file.BadCheckpointException: Configured capacity is 100000000 but the checkpoint file capacity is 1000000. See FileChannel documentation on how to change a channels capacity.
at org.apache.flume.channel.file.EventQueueBackingStoreFile.<init>(EventQueueBackingStoreFile.java:100)
at org.apache.flume.channel.file.EventQueueBackingStoreFileV3.<init>(EventQueueBackingStoreFileV3.java:49)
at org.apache.flume.channel.file.EventQueueBackingStoreFactory.get(EventQueueBackingStoreFactory.java:75)
at org.apache.flume.channel.file.Log.replay(Log.java:412)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:301)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
input {
log4j {
port => 55555
type => flume
}
}
filter {
multiline {
pattern => "^\\s"
what => "previous"
}
}
filter {
mutate {
add_field => [ "source_host", "%{@source_host}" ]
}
}
filter {
grok {
match => [ "message", "%{MONTHDAY:day} %{MONTH:month} %{YEAR:year} %{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second} %{LOGLEVEL:level} \s+\[(?<thread>[a-zA-Z0-9._-]+)\]\s+\((?<culprit>[a-zA-Z0-9.:_-]+)\)\s+- %{GREEDYDATA:message}" ]
overwrite => [ "message" ]
}
grok {
match => [ "message", "%{USERNAME:classname}: %{GREEDYDATA:error}" ]
}
}
output {
elasticsearch {
host => "127.0.0.1"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment