Skip to content

Instantly share code, notes, and snippets.

@dannyk81
Created August 8, 2017 19:12
Show Gist options
  • Save dannyk81/4bb95914509c1e8db6438536e352985f to your computer and use it in GitHub Desktop.
Save dannyk81/4bb95914509c1e8db6438536e352985f to your computer and use it in GitHub Desktop.
Fluentd v0.12 converting long epoch (milliseconds) to Date Time string with milleseconds precision
# Consider the record contains the time stamp of the event in a record key called 'timestamp'
# e.g. "timestamp": "1502217900063"
# The below will add a new record called `formatted_date` that will include an iso8601(3) formatted date string with milliseconds,
# the trick was to extract from the long epoch value the seconds & remaining milliseconds and convert it to microseconds since Time.at() accepts:
# `Time.at(seconds, microseconds_with_frac) → time`
<filter tag.*>
@type record_modifier
<record>
formatted_date ${Time.at(record['timestamp'].to_i/1000, record['timestamp'].to_i%1000*1000).utc.strftime('%Y-%m-%dT%H:%M:%S.%LZ')}
</record>
</filter>
# If you use the fluent-plugin-elasticsearch, you can tell the plugin to use `formatted_date` to generate the @timestamp, like so:
<match **>
type elasticsearch
time_key formatted_date
...
</match>
@vgavro
Copy link

vgavro commented Jun 14, 2021

Comment above loses milliseconds precision. Use time ${record['timestamp'].to_f / 1000} instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment