Skip to content

Instantly share code, notes, and snippets.

@philz
Created August 8, 2019 19:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save philz/193a99909a6e8eaa796b4f8d502afa55 to your computer and use it in GitHub Desktop.
Save philz/193a99909a6e8eaa796b4f8d502afa55 to your computer and use it in GitHub Desktop.
Using jq to compute relative timestamps of JSON log files
#!/bin/bash
#
# Given a log file of JSON objects with timestamps, adds a new field called
# "relativeTimeMs" for the duration between successive log lines, and then
# sorts by that field to pull out the "longest" log lines.
echo '{"time":"2019-08-08T17:08:08.343Z","msg":"foo"}
{"time":"2019-08-08T17:08:08.347Z","msg":"bar"}
{"time":"2019-08-08T17:08:08.349Z","msg":"baz"}
{"time":"2019-08-08T17:08:09.322Z","msg":"booooom"}
' | jq -s '[., .[1:]] | transpose [] | select(.[1].time) |
[ (.[1].time|split(".")[1]|split("Z")[0]|tonumber/1000.0) - (.[0].time|split(".")[1]|split("Z")[0]|tonumber/1000.0) + (.[1].time | split(".")[0] + "Z" | fromdateiso8601) - (.[0].time | split(".")[0] + "Z" | fromdateiso8601) , .[1] ]
| .[1] + { relativeTimeMs: (.[0]*1000) }' \
| jq -r -c '[ .relativeTimeMs, .msg[0:100] ] | @csv' | sort -rn | head
@thuannguyen833
Copy link

echo ' {"time": "2019-08-08T17: 08: 08.343Z", "tin nhắn": "foo"}
{"thời gian": "2019-08-08T17: 08: 08.347Z", "tin nhắn": "thanh"}
{"thời gian": "2019-08-08T17: 08: 08.349Z", "tin nhắn": "baz"}
{"thời gian": "2019-08-08T17: 08: 09.322Z", "tin nhắn": "booooom"}
' | jq -s ' [.,. [1:]] | chuyển vị [] | chọn (. [1] .time) |

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment