Skip to content

Instantly share code, notes, and snippets.

@epcim
Forked from dannyk81/fluentd.conf
Created October 26, 2017 14:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save epcim/93cd47849daec42dc2dc64cb5c08fd50 to your computer and use it in GitHub Desktop.
Save epcim/93cd47849daec42dc2dc64cb5c08fd50 to your computer and use it in GitHub Desktop.
Fluentd v0.12 converting long epoch (milliseconds) to Date Time string with milleseconds precision
# Consider the record contains the time stamp of the event in a record key called 'timestamp'
# e.g. "timestamp": "1502217900063"
# The below will add a new record called `formatted_date` that will include an iso8601(3) formatted date string with milliseconds,
# the trick was to extract from the long epoch value the seconds & remaining milliseconds and convert it to microseconds since Time.at() accepts:
# `Time.at(seconds, microseconds_with_frac) → time`
<filter tag.*>
@type record_modifier
<record>
formatted_date ${Time.at(record['timestamp'].to_i/1000, record['timestamp'].to_i%1000*1000).utc.strftime('%Y-%m-%dT%H:%M:%S.%LZ')}
</record>
</filter>
# If you use the fluent-plugin-elasticsearch, you can tell the plugin to use `formatted_date` to generate the @timestamp, like so:
<match **>
type elasticsearch
time_key formatted_date
...
</match>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment