Skip to content

Instantly share code, notes, and snippets.

@soheilpro
Last active August 3, 2022 18:42
Show Gist options
  • Save soheilpro/a34957550b1bd7d42be2 to your computer and use it in GitHub Desktop.
Save soheilpro/a34957550b1bd7d42be2 to your computer and use it in GitHub Desktop.
Easy IIS log file format specification for goaccess.
goaccess -f u_ex150629.log --log-format "$(cat u_ex150629.log | ./goiisformat.sh)" --date-format '%Y-%m-%d' --time-format '%H:%M:%S'
#!/usr/bin/env sh
while read line; do
if [[ $line == \#Fields:* ]]; then
line=${line/\#Fields: /}
line=${line/date/%d}
line=${line/time/%t}
line=${line/s-sitename/%^}
line=${line/s-computername/%^}
line=${line/s-ip/%^}
line=${line/cs-method/%m}
line=${line/cs-uri-stem/%U}
line=${line/cs-uri-query/%^}
line=${line/s-port/%^}
line=${line/cs-username/%^}
line=${line/c-ip/%h}
line=${line/cs-version/%H}
line=${line/cs(User-Agent)/%u}
line=${line/cs(Cookie)/%^}
line=${line/cs(Referer)/%R}
line=${line/cs-host/%^}
line=${line/sc-status/%s}
line=${line/sc-substatus/%^}
line=${line/sc-win32-status/%^}
line=${line/sc-bytes/%b}
line=${line/cs-bytes/%^}
line=${line/time-taken/%L}
echo $line
exit;
fi
done
@felquis
Copy link

felquis commented Apr 5, 2016

My goiisformat.sh became

#!/usr/bin/env sh

while read line; do
  if [[ $line == \#Fields:* ]]; then
    line=${line/\#Fields: /}
    line=${line/date/%d}
    line=${line/time/%t}
    line=${line/s-sitename/%^}
    line=${line/cs-method/%m}
    line=${line/cs-uri-stem/%U}
    line=${line/cs-uri-query/%^}
    line=${line/s-port/%^}
    line=${line/cs-username/%^}
    line=${line/c-ip/%h}
    line=${line/cs(User-Agent)/%u}
    line=${line/cs(Cookie)/%^}
    line=${line/cs(Referer)/%R}
    line=${line/cs-host/%^}
    line=${line/sc-status/%s}
    line=${line/sc-substatus/%^}
    line=${line/sc-win32-status/%^}
    line=${line/sc-bytes/%b}
    line=${line/cs-bytes/%^}
    line=${line/time-taken/%L}
    echo $line
    exit;
  fi
done

And I had to remove the --time-format value to make it work.. based on this comment allinurl/goaccess#233 (comment)

@elproducto
Copy link

elproducto commented Jan 14, 2017

Thanks to you both for your contribution to help process IIS Log files in GoAccess. You have inspired me to contribute as well. I am using the following syntax to process multiple IIS file. However all my logs prior to 2016 are throwing error when using the this syntax, error stating content does not match specifier.

I decided to combine my logs into a single W3C file using Microsoft Log Parser, however to process the file I needed to modify the goiisformat script to work with file outputted by the Parser. I have shared the modifications in a forked script gomspformat.sh . The gomspformat script will provide specifiers for W3C file generated by Microsoft Log Parser. I used the Log parser to combine my IIS logs into a single W3C file, then used GoAccess and the gomspformat to process this file.

I wonder if this method would work to process multiple Server IIS logs into a single report.

Multiple IIS Logs via GoAccess and any Soheilpro's GoIISFormat Script

cat  u_ex* | goaccess --log-format "$(cat u_ex170114.log | ~/goiisformat.sh)" --date-format '%Y-%m-%d' --time-format '%H:%M:%S'

In this example a random file was selected for the string "cat u_ex170114.log" to be piped to the script
Just as an FYI you will need to ensure that the goiisformat.sh script created is not in DOS format. Dos formatted files will throw an error message with "command not found". I used Nano inside of Cygwin to make the goiishformat.sh script file.

@henrikj242
Copy link

I've just been asked to analyze some old IIS log files. The web server, an IIS v. 6 was configured to log using the W3C extended format.

I ended up using goaccess like this:
goaccess -f all.log --log-format '%d %t %^ %^ %m %U %^ %^ %^ %h %u %s %^ %^' --date-format '%Y-%m-%d' --time-format '%H:%M:%S'

@Sarkie
Copy link

Sarkie commented Dec 9, 2019

another example for anyone
goaccess u_ex191124.log --log-format '%d %t %^ %m %r - %^ - %h %u %s %^ %T' --date-format '%Y-%m-%d' --time-format '%H:%M:%S'

@Simran17032023
Copy link

I am facing an error src/parser.c - read_log - 3297
Can anyone help me to pass the iis logs ,Please its urgent...

@akhansari
Copy link

My 2 cents
goaccess --log-format '%d %t %^ %v %^ %m %U %q %^ %e %h %^ %u %R %^ %s %^ %^ %b %^ %L' --date-format '%Y-%m-%d' --time-format '%H:%M:%S' -o stats.html -f *.log
for the following header
date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment