Skip to content

Instantly share code, notes, and snippets.

@guitarrapc
Last active January 24, 2019 02:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save guitarrapc/afcfef635e1f86012ba61a50e4cc4cd9 to your computer and use it in GitHub Desktop.
Save guitarrapc/afcfef635e1f86012ba61a50e4cc4cd9 to your computer and use it in GitHub Desktop.
Import-Csv in PowerShell 6.0 supports w3c log format, if delimiter is `,`. https://github.com/PowerShell/PowerShell/pull/2482
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 1 column, instead of 3. in line 4.
#Software: Microsoft Exchange Server
#Version: 15.0.0.0
#Log-type: Transport Connectivity Log
#Date: 2016-09-16T23:30:07.338Z
#Fields: Column1,Column2,Column 3
data1,1,A
data2,2,B
data3,3,C
data4,4,D
#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2013-06-24 10:56:45
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
2013-06-24 10:56:45 192.168.0.1 POST /xas/ - 80 - 222.222.222.222 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/27.0.1453.110+Safari/537.36 404 0 2 471
<!-- just for logFile section -->
<!-- there's no attribute, value to change delimiter -->
<element name="logFile">
<attribute name="logExtFileFlags" type="flags" defaultValue="Date, Time, ClientIP, UserName, ServerIP, Method, UriStem, UriQuery, TimeTaken, HttpStatus, Win32Status, ServerPort, UserAgent, HttpSubStatus, Referer">
<flag name="Date" value="1"/>
<flag name="Time" value="2"/>
<flag name="ClientIP" value="4"/>
<flag name="UserName" value="8"/>
<flag name="SiteName" value="16"/>
<flag name="ComputerName" value="32"/>
<flag name="ServerIP" value="64"/>
<flag name="Method" value="128"/>
<flag name="UriStem" value="256"/>
<flag name="UriQuery" value="512"/>
<flag name="HttpStatus" value="1024"/>
<flag name="Win32Status" value="2048"/>
<flag name="BytesSent" value="4096"/>
<flag name="BytesRecv" value="8192"/>
<flag name="TimeTaken" value="16384"/>
<flag name="ServerPort" value="32768"/>
<flag name="UserAgent" value="65536"/>
<flag name="Cookie" value="131072"/>
<flag name="Referer" value="262144"/>
<flag name="ProtocolVersion" value="524288"/>
<flag name="Host" value="1048576"/>
<flag name="HttpSubStatus" value="2097152"/>
</attribute>
<attribute name="customLogPluginClsid" type="string" defaultValue=""/>
<attribute name="logFormat" type="enum" defaultValue="W3C">
<enum name="IIS" value="0"/>
<enum name="NCSA" value="1"/>
<enum name="W3C" value="2"/>
<enum name="Custom" value="3"/>
</attribute>
<attribute name="logTargetW3C" type="flags" defaultValue="File">
<flag name="File" value="1"/>
<flag name="ETW" value="2"/>
</attribute>
<attribute name="directory" type="string" expanded="true" defaultValue="%SystemDrive%\inetpub\logs\LogFiles" validationType="nonEmptyString" />
<attribute name="period" type="enum" defaultValue="Daily">
<enum name="MaxSize" value="0"/>
<enum name="Daily" value="1"/>
<enum name="Weekly" value="2"/>
<enum name="Monthly" value="3"/>
<enum name="Hourly" value="4"/>
</attribute>
<attribute name="truncateSize" type="int64" defaultValue="20971520" validationType="integerRange" validationParameter="1048576,4294967295" />
<attribute name="localTimeRollover" type="bool" defaultValue="false"/>
<attribute name="enabled" type="bool" defaultValue="true" />
<attribute name="logSiteId" type="bool" defaultValue="true" />
<attribute name="flushByEntryCountW3CLog" type="uint" defaultValue="0" />
<attribute name="maxLogLineLength" type="uint" validationType="integerRange" validationParameter="2,65536" defaultValue="65536" />
<element name="customFields">
<attribute name="maxCustomFieldLength" type="uint" validationType="integerRange" validationParameter="2,65536" defaultValue="4096" />
<collection addElement="add" clearElement="clear">
<attribute name="logFieldName" type="string" required="true" isUniqueKey="true" validationType="nonEmptyString" />
<attribute name="sourceName" type="string" required="true" validationType="nonEmptyString" />
<attribute name="sourceType" type="enum" required="true" >
<enum name="RequestHeader" value="0"/>
<enum name="ResponseHeader" value="1"/>
<enum name="ServerVariable" value="2"/>
</attribute>
</collection>
</element>
</element>
<element name="traceFailedRequestsLogging">
<attribute name="enabled" type="bool" defaultValue="false" />
<attribute name="directory" type="string" expanded="true" defaultValue="%SystemDrive%\inetpub\logs\FailedReqLogFiles"/>
<attribute name="maxLogFiles" type="uint" defaultValue="50" validationType="integerRange" validationParameter="1,10000"/>
<attribute name="maxLogFileSizeKB" type="uint" defaultValue="1024" validationType="integerRange" validationParameter="0,1048576"/>
<attribute name="customActionsEnabled" type="bool" defaultValue="false"/>
</element>
import-csv .\iis_log.log
<#
date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
------------------------------------------------------------------------------------------------------------------------------------------
2013-06-24 10:56:45 192.168.0.1 POST /xas/ - 80 - 222.222.222.222 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML
#>
import-csv .\iis_log.csv
# https://github.com/iSazonov/PowerShell/blob/0818b6c921c1970dc294669134266f878352891a/test/powershell/Modules/Microsoft.PowerShell.Utility/assets/TestImportCsv_W3C_ELF.csv
<#
Column1 Column2 Column 3
------- ------- --------
data1 1 A
data2 2 B
data3 3 C
data4 4 D
#>
@guitarrapc
Copy link
Author

conslusion

don't think about iis log. But this Import-Csv enhancement will save some w3c log output with delimiter "," product.

description

This change is begin from following issue.

PowerShell/PowerShell#2480

w3c log format is not only for IIS, and IIS could not change log delimiter with schema or web.config.

https://stackoverflow.com/questions/35847585/cant-change-iis-log-field-delimter

not sure about advanced logging, but may possible....?

https://forums.iis.net/t/1191687.aspx?Change+log+file+delimiter+

@guitarrapc
Copy link
Author

IIS w3c log, delimited via space, is one of the most unlike format I've ever seen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment