JSON from varnishncsa

varnishncsa.json

varnishncsa -F '{
    "@source" => "unknown",
    "@type" => "varnish",
    "@fields" => {
        "remote_addr"=>"%h",
        "remote_user" => "%u",
        "x_forwarded_for" => "%{X-Forwarded-For}i",
        "hit_miss"=>" %{Varnish:hitmiss}x",
        "body_bytes_sent"=>"%b",
        "request_time"=>"%{Varnish:time_firstbyte}x",
        "status"=>"%s",
        "request"=>"%r",
        "host"=>"%{host}i",
        "request_method"=>"%m",
        "time_first_byte" => "%{Varnish:time_firstbyte}x",
        "handling" => "%{Varnish:handling}x",
        "http_referrer"=>"%{Referrer}i",
        "http_user_agent"=>"%{User-agent}i",
    }
}'

If you want to do it logstash style, you can do:

sudo varnishncsa -F '{
    "@timestamp": "%{%Y-%m-%dT%H:%M:%S%z}t",
    "@fields" : {
        "client":"%h",
        "remote_user" : "%u",
        "x_forwarded_for" : "%{X-Forwarded-For}i",
        "hit_miss":" %{Varnish:hitmiss}x",
        "bytes": %b,
        "duration_usec": %D,
        "status": %s,
        "request":"%r",
        "virtualhost":"%{host}i",
        "method":"%m",
        "time_first_byte" : "%{Varnish:time_firstbyte}x",
        "handling" : "%{Varnish:handling}x",
        "referrer":"%{Referrer}i",
        "user_agent":"%{User-agent}i",
    }
}'

I've also renamed some of the fields

remote_addr         -> client
body_bytes_sent     -> bytes
request_time        -> duration_usec
host                -> virtualhost
time_first_byte     -> #removed
http_method         -> method
http_referrer       -> referrer
http_user_agent     -> user_agent

To better match the Apache JSON Log Format here:

https://github.com/logstash/cookbook/blob/gh-pages/recipes/apache-json-logs/apache.conf#L5