How to map logs received from multiple hosts on the same local port to the correct sourcetype and index.
The Splunk data input UDP:514 receives events from the following devices:
10.0.0.1
: A Netfilter firewall10.0.0.2
: A Squid proxy instance (1/2)10.0.0.3
: A Squid proxy instance (2/2)
We want to separate logs comming from the Firewall (10.0.0.1
) and the Proxies (10.0.0.2
and 10.0.0.3
):
- Firewall logs (
10.0.0.1
) in their own index (firewall
) and sourcetype (netfilter
) - Proxies logs (
10.0.0.2
and10.0.0.3
) in their common index (proxy
) and sourcetype (squid
)
[10.0.0.1] ---> UDP:514 --\ /--> index: firewall, sourcetype: netfilter
[10.0.0.2] ---> UDP:514 ---> [Splunk] ---> [inputs.conf] ---> [props.conf] ---> [transforms.conf]
[10.0.0.3] ---> UDP:514 --/ \--> index: proxy, sourcetype: squid
We define a root input which receive logs on port UDP:514 and forward the events to the index syslog
with the sourcetype syslog
.
[udp://514]
index = syslog
sourcetype = syslog
We define one stanza for each source or source group (using the source IP or hostname).
[host::10.0.0.1]
TRANSFORMS-netfilter = index_firewall, sourcetype_netfilter
[host::10.0.0.(2|3)]
TRANSFORMS-squid = index_proxy, sourcetype_squid
We define two stanzas per source: one for the index, one for the sourcetype. Notice the leading underscore _MetaData
in the index_ stanzas, and not for the sourcetype_ stanzas.
[index_firewall]
REGEX = .*
DEST_KEY = _MetaData:Index
FORMAT = firewall
[sourcetype_netfilter]
REGEX = .*
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::netfilter
[index_proxy]
REGEX = .*
DEST_KEY = _MetaData:Index
FORMAT = proxy
[sourcetype_squid]
REGEX = .*
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::squid
Updated document using real-world sources (Squid and Netfilter).