Skip to content

Instantly share code, notes, and snippets.

@guitarrapc guitarrapc/IIS_Schema.xml
Last active Jan 24, 2019

Embed
What would you like to do?
Import-Csv in PowerShell 6.0 supports w3c log format, if delimiter is `,`. https://github.com/PowerShell/PowerShell/pull/2482
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 1 column, instead of 3. in line 4.
#Software: Microsoft Exchange Server
#Version: 15.0.0.0
#Log-type: Transport Connectivity Log
#Date: 2016-09-16T23:30:07.338Z
#Fields: Column1,Column2,Column 3
data1,1,A
data2,2,B
data3,3,C
data4,4,D
#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2013-06-24 10:56:45
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
2013-06-24 10:56:45 192.168.0.1 POST /xas/ - 80 - 222.222.222.222 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/27.0.1453.110+Safari/537.36 404 0 2 471
<!-- just for logFile section -->
<!-- there's no attribute, value to change delimiter -->
<element name="logFile">
<attribute name="logExtFileFlags" type="flags" defaultValue="Date, Time, ClientIP, UserName, ServerIP, Method, UriStem, UriQuery, TimeTaken, HttpStatus, Win32Status, ServerPort, UserAgent, HttpSubStatus, Referer">
<flag name="Date" value="1"/>
<flag name="Time" value="2"/>
<flag name="ClientIP" value="4"/>
<flag name="UserName" value="8"/>
<flag name="SiteName" value="16"/>
<flag name="ComputerName" value="32"/>
<flag name="ServerIP" value="64"/>
<flag name="Method" value="128"/>
<flag name="UriStem" value="256"/>
<flag name="UriQuery" value="512"/>
<flag name="HttpStatus" value="1024"/>
<flag name="Win32Status" value="2048"/>
<flag name="BytesSent" value="4096"/>
<flag name="BytesRecv" value="8192"/>
<flag name="TimeTaken" value="16384"/>
<flag name="ServerPort" value="32768"/>
<flag name="UserAgent" value="65536"/>
<flag name="Cookie" value="131072"/>
<flag name="Referer" value="262144"/>
<flag name="ProtocolVersion" value="524288"/>
<flag name="Host" value="1048576"/>
<flag name="HttpSubStatus" value="2097152"/>
</attribute>
<attribute name="customLogPluginClsid" type="string" defaultValue=""/>
<attribute name="logFormat" type="enum" defaultValue="W3C">
<enum name="IIS" value="0"/>
<enum name="NCSA" value="1"/>
<enum name="W3C" value="2"/>
<enum name="Custom" value="3"/>
</attribute>
<attribute name="logTargetW3C" type="flags" defaultValue="File">
<flag name="File" value="1"/>
<flag name="ETW" value="2"/>
</attribute>
<attribute name="directory" type="string" expanded="true" defaultValue="%SystemDrive%\inetpub\logs\LogFiles" validationType="nonEmptyString" />
<attribute name="period" type="enum" defaultValue="Daily">
<enum name="MaxSize" value="0"/>
<enum name="Daily" value="1"/>
<enum name="Weekly" value="2"/>
<enum name="Monthly" value="3"/>
<enum name="Hourly" value="4"/>
</attribute>
<attribute name="truncateSize" type="int64" defaultValue="20971520" validationType="integerRange" validationParameter="1048576,4294967295" />
<attribute name="localTimeRollover" type="bool" defaultValue="false"/>
<attribute name="enabled" type="bool" defaultValue="true" />
<attribute name="logSiteId" type="bool" defaultValue="true" />
<attribute name="flushByEntryCountW3CLog" type="uint" defaultValue="0" />
<attribute name="maxLogLineLength" type="uint" validationType="integerRange" validationParameter="2,65536" defaultValue="65536" />
<element name="customFields">
<attribute name="maxCustomFieldLength" type="uint" validationType="integerRange" validationParameter="2,65536" defaultValue="4096" />
<collection addElement="add" clearElement="clear">
<attribute name="logFieldName" type="string" required="true" isUniqueKey="true" validationType="nonEmptyString" />
<attribute name="sourceName" type="string" required="true" validationType="nonEmptyString" />
<attribute name="sourceType" type="enum" required="true" >
<enum name="RequestHeader" value="0"/>
<enum name="ResponseHeader" value="1"/>
<enum name="ServerVariable" value="2"/>
</attribute>
</collection>
</element>
</element>
<element name="traceFailedRequestsLogging">
<attribute name="enabled" type="bool" defaultValue="false" />
<attribute name="directory" type="string" expanded="true" defaultValue="%SystemDrive%\inetpub\logs\FailedReqLogFiles"/>
<attribute name="maxLogFiles" type="uint" defaultValue="50" validationType="integerRange" validationParameter="1,10000"/>
<attribute name="maxLogFileSizeKB" type="uint" defaultValue="1024" validationType="integerRange" validationParameter="0,1048576"/>
<attribute name="customActionsEnabled" type="bool" defaultValue="false"/>
</element>
import-csv .\iis_log.log
<#
date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status time-taken
------------------------------------------------------------------------------------------------------------------------------------------
2013-06-24 10:56:45 192.168.0.1 POST /xas/ - 80 - 222.222.222.222 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.36+(KHTML
#>
import-csv .\iis_log.csv
# https://github.com/iSazonov/PowerShell/blob/0818b6c921c1970dc294669134266f878352891a/test/powershell/Modules/Microsoft.PowerShell.Utility/assets/TestImportCsv_W3C_ELF.csv
<#
Column1 Column2 Column 3
------- ------- --------
data1 1 A
data2 2 B
data3 3 C
data4 4 D
#>
@guitarrapc

This comment has been minimized.

Copy link
Owner Author

guitarrapc commented Jan 24, 2019

conslusion

don't think about iis log. But this Import-Csv enhancement will save some w3c log output with delimiter "," product.

description

This change is begin from following issue.

PowerShell/PowerShell#2480

w3c log format is not only for IIS, and IIS could not change log delimiter with schema or web.config.

https://stackoverflow.com/questions/35847585/cant-change-iis-log-field-delimter

not sure about advanced logging, but may possible....?

https://forums.iis.net/t/1191687.aspx?Change+log+file+delimiter+

@guitarrapc

This comment has been minimized.

Copy link
Owner Author

guitarrapc commented Jan 24, 2019

IIS w3c log, delimited via space, is one of the most unlike format I've ever seen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.