Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save JonasHaouzi/e313ca7bb567f7274536441aa5880ff7 to your computer and use it in GitHub Desktop.
Save JonasHaouzi/e313ca7bb567f7274536441aa5880ff7 to your computer and use it in GitHub Desktop.
fields @timestamp, @message
| sort @userAgent desc
| parse '* (*) - * [*] "* * *" * * "*" "*"' as localIp, realIp, identity, dateTimeString, httpVerb, url, protocol, statusCode, bytes, referer, userAgent
| filter @message like /Googlebot/ and @realIp = '66.249.69.166'
fields @timestamp, @message
| sort @timestamp desc
| parse '* (*) - * [*] "* * *" * * "*" "*"' as localIp, realIp, identity, dateTimeString, httpVerb, url, protocol, statusCode, bytes, referer, userAgent
| filter @message like /Googlebot/
fields @timestamp, @message
| sort @timestamp desc
| parse '* (*) - * [*] "* * *" * * "*" "*"' as localIp, realIp, identity, dateTimeString, httpVerb, url, protocol, statusCode, bytes, referer, userAgent
| filter @message like /Googlebot/
stats count(*) as numRequests by url
| sort @timestamp desc
| parse '* (*) - * [*] "* * *" * * "*" "*"' as localIp, realIp, identity, dateTimeString, httpVerb, url, protocol, statusCode, bytes, referer, userAgent
| filter @message like /Googlebot/
fields @timestamp, @message
| sort @timestamp desc
| parse '* (*) - * [*] "* * *" * * "*" "*"' as localIp, realIp, identity, dateTimeString, httpVerb, url, protocol, statusCode, bytes, referer, userAgent
| filter @message like /Googlebot/ and @message not like /\.(js|css|jpg|png|jpeg|gif|ico|svg)/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment