Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
NXlog config to create a syslog server on Windows

NXlog config to create a syslog server on Windows

Creates a log file for each connecting syslog client, based on IP address. Also takes care of rotating the files, limiting the archive to 5 log files of 100M This config is meant to allow a Splunk Universal Forwarder to collect the syslog files, using the following inputs.conf:

inputs.conf (Splunk):

[monitor://c:/log/192.168.1.1/*.log]
index = cisco
sourcetype = cisco:asa
disabled = false
host_segment = 3

nxlog.conf (NXlog-CE):

define ROOT C:\Program Files (x86)\nxlog

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

<Extension syslog>
    Module  xm_syslog
</Extension>

<Extension fileop>
    Module  xm_fileop
</Extension>

<Input in>
    Module  im_tcp
    Host  0.0.0.0
    Port  514
    Exec  parse_syslog_ietf();
</Input>

<Output out>
    Module  om_file
    File  "Z:/nxlog/" + $MessageSourceAddress + "/" + $MessageSourceAddress + "-" + $Severity + ".log"
    CreateDir TRUE
    Exec if (out->file_size() > 100M ) file_cycle("Z:/nxlog/" + $MessageSourceAddress + "/" + $MessageSourceAddress + "-" + $Severity + ".log",5);
    Exec out->reopen();
</Output>

<Route 1>
    Path  in => out
</Route>

Performance

NXlog seem to max out at 4000 syslog events per second or 8.4 Mbit per second. On an AWS m3.large instance with 2 vCPU's and 30GB local SSD the following observations were made:

Eps Mbps CPU %
500 1.0 7
1000 2.1 15
2000 4.2 26
4000 8.4 50

Errors

Note that when monitoring log files written bt NXlog, Splunkd will complain with errors like these below, however it doesn't lose any messages. The number of lines over 3 rotated logfiles equals the number of indexed events in Splunk. Good times!

04-17-2016 12:39:06.265 +0000 ERROR TailReader - error from read call from 'Z:\nxlog\172.31.9.1\172.31.9.1-INFO.log'.
04-17-2016 12:39:06.265 +0000 ERROR WatchedFile - Error reading file 'Z:\nxlog\172.31.9.1\172.31.9.1-INFO.log'. ErrorCode = 33
04-17-2016 12:38:33.203 +0000 ERROR TailReader - failed to compute crc for Z:\nxlog\172.31.9.1\172.31.9.1-INFO.log (method: 0, hint: The process cannot access the file because another process has locked a portion of the file.).
04-17-2016 12:38:33.203 +0000 WARN WatchedFile - encountered error computing crc, hint: [seekptr=52290551,start_read=52290295,read_sz=256] (The process cannot access the file because another process has locked a portion of the file.)
04-17-2016 12:38:33.203 +0000 WARN FileInputTracker - Error reading CRC: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:53.171 +0000 ERROR TailReader - Error while reading data for initCrc: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:22.162 +0000 ERROR TailReader - Error while reading data for initCrc: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:22.162 +0000 ERROR TailReader - failed to compute crc for Z:\nxlog\172.31.9.1\172.31.9.1-INFO.log (method: 0, hint: The process cannot access the file because another process has locked a portion of the file.).
04-17-2016 12:37:22.161 +0000 WARN WatchedFile - encountered error computing crc, hint: [seekptr=22231540,start_read=22231284,read_sz=256] (The process cannot access the file because another process has locked a portion of the file.)
04-17-2016 12:37:22.161 +0000 WARN FileInputTracker - Error reading CRC: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:21.140 +0000 ERROR TailReader - Error while reading data for initCrc: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:21.140 +0000 ERROR TailReader - Error while reading data for initCrc: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:21.140 +0000 ERROR TailReader - failed to compute crc for Z:\nxlog\172.31.9.1\172.31.9.1-INFO.log (method: 0, hint: The process cannot access the file because another process has locked a portion of the file.).
04-17-2016 12:37:21.140 +0000 WARN WatchedFile - encountered error computing crc, hint: [seekptr=21314036,start_read=21313780,read_sz=256] (The process cannot access the file because another process has locked a portion of the file.)
04-17-2016 12:37:21.140 +0000 WARN FileInputTracker - Error reading CRC: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:21.131 +0000 ERROR TailReader - Failed to compute seek crc
04-17-2016 12:37:21.131 +0000 WARN WatchedFile - encountered error computing crc, hint: [seekptr=21248500,start_read=21248244,read_sz=256] (The process cannot access the file because another process has locked a portion of the file.)
04-17-2016 12:37:21.131 +0000 WARN FileInputTracker - Error reading CRC: The process cannot access the file because another process has locked a portion of the file.
04-17-2016 12:37:12.123 +0000 ERROR TailReader - Failed to compute seek crc

Note that if you forget to include the out->reopen() statement in nxlog.conf, you will get error messages like this in nxlog.log:

2016-04-17 11:49:43 INFO connection accepted from 172.31.9.1:55423
2016-04-17 11:52:25 INFO removing file Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log.3
2016-04-17 11:52:25 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:28 ERROR last message repeated 6857 times
2016-04-17 11:52:28 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:31 ERROR last message repeated 6829 times
2016-04-17 11:52:31 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:34 ERROR last message repeated 6837 times
2016-04-17 11:52:34 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:37 ERROR last message repeated 6823 times
2016-04-17 11:52:37 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:40 ERROR last message repeated 6802 times
2016-04-17 11:52:40 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:43 ERROR last message repeated 6809 times
2016-04-17 11:52:43 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:46 ERROR last message repeated 6804 times
2016-04-17 11:52:46 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:49 ERROR last message repeated 6812 times
2016-04-17 11:52:49 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:52 ERROR last message repeated 6788 times
2016-04-17 11:52:52 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:55 ERROR last message repeated 6822 times
2016-04-17 11:52:55 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:52:58 ERROR last message repeated 6813 times
2016-04-17 11:52:58 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:53:01 ERROR last message repeated 6037 times
2016-04-17 11:53:01 ERROR if-else failed at line 29, character 142 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; binary operation failed at line 29, character 34 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; function 'file_size' failed at line 29, character 25 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. expression evaluation has been aborted; failed to query file size information for Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log; The system cannot find the file specified.  
2016-04-17 11:56:34 ERROR last message repeated 1143 times
2016-04-17 11:56:34 INFO removing file Z:/nxlog/172.31.9.1/172.31.9.1-INFO.log.3
@JohnCrawford4587

This comment has been minimized.

Copy link

@JohnCrawford4587 JohnCrawford4587 commented Apr 5, 2017

One thing I noticed that is intermittent on the NXLOG server:

2017-04-05 15:27:02 ERROR procedure 'reopen' failed at line 42, character 23 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement execution has been aborted; failed to open ; The system cannot find the path specified.

Line 42 in my case is the Exec out->reopen();. I'm still investigating and trying to get to the bottom of it and will post back on my findings later. I'm brand new to NXLog, so it may be a while.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.