Skip to content

Instantly share code, notes, and snippets.

@mbierman
Last active February 28, 2024 21:55
Show Gist options
  • Star 21 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save mbierman/f3d184b65e0f4de6fa75a4a5d5145426 to your computer and use it in GitHub Desktop.
Save mbierman/f3d184b65e0f4de6fa75a4a5d5145426 to your computer and use it in GitHub Desktop.
Add a remote syslog server to Firewalla
#!/bin/bash
# v 2.1.0
syslog=/etc/rsyslog.d/09-externalserver.conf
# this logs notice and above. use *.* log everything.
filter=*.notice
server=192.168.0.19 # Change the server to the IP of your syslog server.
port=514
hostname=firewalla
valid=$(grep "$server:$port" $syslog 2>/dev/null)
create () {
# To use TCP uncomment line 13 to use TCP and comment line 15
# echo -e "# remote syslog server (TCP):\n$filter @@$server:$port" | sudo tee $syslog
# Line 15 assumes UDP: to use TCP, comment the line 15 and uncomment line 13
echo -e "# remote syslog server (UDP):\n\$LocalHostName $hostname\nfilter @$server:$port" | sudo tee $syslog
echo "Restarting rsyslog..."
sudo systemctl restart rsyslog
echo "remote syslog added"
exit
}
cleanup () {
sudo rm -f $syslog
sudo systemctl restart rsyslog
}
if [ -f "$syslog" ] ; then
if [ -n "$valid" ] ; then
echo "remote syslog already in place with $server:$port specified"
case $1 in
-c)
echo -e "\nrecreating syslog configuration..."
cleanup
create
;;
-r|-restart|-force|-f)
echo "Restarting rsyslog..."
sudo systemctl restart rsyslog
exit
;;
-u|-update)
read -p "Are you sure you want to remove the syslog forwarder? type 'y' " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]] ; then
ls $syslog 2>/dev/null && cleanup || echo -e "\n\nNo log found.\n"
fi
exit
;;
-h)
echo -e "You can use:\n - \`$0 -c\` recreate forwarding\n - \`$0 -r\` restart the syslog service\
\n - \`$0 -u\` uninstall the settings to send to the remote syslog server\n\n"
exit
;;
esac
else
echo "The server is not configured correctly. On it."
cleanup
create
fi
else
echo "There was no syslog forwarder in place."
create
fi
@mbierman
Copy link
Author

mbierman commented Apr 3, 2022

Instructions

To send logs to a remote syslog server using UDP, do the following:

  1. ssh to the Firewalla box.
  2. Copy the script above.
  3. If this directory doesn’t exist, create it first.
sudo mkdir /home/pi/.firewalla/config/post_main.d/
  1. Then create the file:
sudo vi /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh
  1. Paste this script into the file you just created. This is going to persist the syslog setting even if there's a firewalla update that wipes out the settings in the future.
  2. Edit the server to the IP address of your syslog server.
  3. Save the file.
  4. Give the script execute permissions.
    sudo chmod a+x /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh
  5. Execute the script.
    /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh creates the file and restarts syslog
    /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh -r restarts syslog
    /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh -u uninstalls the forwarder and restarts syslog.

Notes

  • Obviously, you need a remote syslog server set up and running for this to work. See step 6. You will need to match the port you set here with the port the syslog server accepts traffic from.
  • This will persist if your Firewalla is upgraded or rebooted.
  • I decided to filter “notice” and above. “Info” was too noisy for my needs. You can change the logging level if you wish. Replace “*.info” with “*.*” to log everything.
  • There test to see if the file is already in place. If it exists, no need to restart rsyslog.
  • The files in the directory execute in alpha order and some have "stop" commands, so this always executes.

Thanks to Nex_ISS. and Rich T for inspiration.

@whatever152
Copy link

Hey Michael. Thank you for taking the time to build this! I wanted to point out that the script name you have is "addreomotesyslog.sh", but it should be "addremotesyslog.sh".

@mbierman
Copy link
Author

Hey Michael. Thank you for taking the time to build this! I wanted to point out that the script name you have is "addreomotesyslog.sh", but it should be "addremotesyslog.sh".

Good catch! Thanks.

@C0ntr07
Copy link

C0ntr07 commented Apr 13, 2022

Nicely done! Thank you.

@mbierman
Copy link
Author

v 2.0 Small update to clean things up a little bit.

@Karma1331
Copy link

Great script! I believe line 14 should be re-read and fixed to say comment TCP line and uncomment UDP line :)

@mbierman
Copy link
Author

Great script! I believe line 14 should be re-read and fixed to say comment TCP line and uncomment UDP line :)

Updated the comments. Let me know if you think that is more clear.

@pritchey
Copy link

pritchey commented Dec 14, 2022

Thank you very much for this. I do have one minor tweak to offer: As it currently transmits the log entries, they are all originating from the hostname "localhost". This gets confusing on the receiving end - my little tweak let's me define the hostname I want to use (such as "firewalla" and then add an additional line to the 09-externalserver.conf file:

  1. Add this line up where filter, server and port are defined:

hostname=firewalla

  1. Modify line 15 in the script (which echos the config material into the conf file for UDP transmission) to the following:

echo -e "# remote syslog server (UDP):\n\$LocalHostName $hostname\nfilter @$server:$port" | sudo tee $syslog

Notice I injected the "$LocalHostName $hostname\n" portion resulting in the file looking like the following:

# remote syslog server (UDP):
$LocalHostName firewalla
filter @192.168.1.12:514

(Notice the "\" before the $ is needed to escape it so an actual $ is echoed out and variable substitution isn't performed.)

@saiful0190
Copy link

saiful0190 commented May 11, 2023

Hello as instructed ,followed all the steps and i didnt get any logs in syslog server. may i know how can i fix the issue?

  1. Do you have a working syslog server?
  2. Did you set the IP of that syslog server in the script?

@mbierman
Copy link
Author

Thank you very much for this. I do have one minor tweak to offer: As it currently transmits the log entries, they are all originating from the hostname "localhost". This gets confusing on the receiving end - my little tweak let's me define the hostname I want to use (such as "firewalla" and then add an additional line to the 09-externalserver.conf file:

  1. Add this line up where filter, server and port are defined:

hostname=firewalla

  1. Modify line 15 in the script (which echos the config material into the conf file for UDP transmission) to the following:

echo -e "# remote syslog server (UDP):\n\$LocalHostName $hostname\nfilter @$server:$port" | sudo tee $syslog

Notice I injected the "$LocalHostName $hostname\n" portion resulting in the file looking like the following:

# remote syslog server (UDP):
$LocalHostName firewalla
filter @192.168.1.12:514

(Notice the "" before the $ is needed to escape it so an actual $ is echoed out and variable substitution isn't performed.)

Integrated. Great ideas! Thank you.

@RayBishopTN
Copy link

Hello, is it possible to send /log/blog to the remote syslog server? I would like to see the src and dst IP from the /log/blog/current/conn.log

@mjaestewart
Copy link

mjaestewart commented Dec 9, 2023

Hi All!

While I do appreciate great scripting abilities as it makes any process more simple, I decided that the *.* @server:port is the legacy way of collecting logs, where adopting the new rsyslog parms is more powerful and gives us greater flexibility.

The problem with the current scripting method is that it doesn't capture all the logs that we actually really want from Firewalla. The *.* captures everything that's written in the /var/log/syslog directory. Very useful but lacks quite a bit of networking data that we want to see from the /bspool/manager and the /alog/firewalla/ dir (ie conn.log, conn_long, etc). The reason we don't see the connection logs is because they aren't written to the /var/log/syslog dir.

By abandoning the legacy rsyslog parameters we can adopt the new system which is more powerful and flexible.

################
Prerequisite
################
Did you setup the syslog inputs on the synology in log center?

If not go to Log Center > Log Receiving > Create > Give your connection a name, then specify whether you want to use TCP or UDP on port 514. BSD format is fine as well.

######################
New Firewalla Syslog Config
######################

# deifne global workDirectory for saving the state file of log messages.
global(workDirectory="/var/spool/rsyslog")

# enable the Rsyslog imfile module processing text files or logs.
module(load="imfile" PollingInterval="10")


# define template for StandardSyslogFormat for processing log messages.
# that will be forwarded to rsyslog server
template(
    name="StandardSyslogFormat"
    type="string"
    string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
    )


# define ruleset "forwardSysLogs" with action object to send logs to rsyslog server
# define the queue
ruleset(name="forwardSysLogs") {
    action(
        type="omfwd"
        target="172.16.2.20"  # set your Synology Syslog Server NAS IP
        port="514"  # Specify port number
        protocol="tcp"  # specify protocol UDP or TCP
        template="StandardSyslogFormat"  # specifies the template to use above

        queue.SpoolDirectory="/var/spool/rsyslog"
        queue.FileName="remote"
        queue.MaxDiskSpace="1g"
        queue.SaveOnShutdown="on"
        queue.Type="LinkedList"
        ResendLastMSGOnReconnect="on"
        )
        stop
}

# define input files forwardSysLogs logs to send to the rsyslog server
# and apply ruleset "forwardSysLogs" 
# in /bspool/manager

input(type="imfile" ruleset="forwardSysLogs" Tag="ConnLog" File="/bspool/manager/conn.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="ConnLongLog" File="/bspool/manager/conn_long.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="DNS" File="/bspool/manager/dns.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Files" File="/bspool/manager/files.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="HeartBeat" File="/bspool/manager/heartbeat.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="NTP" File="/bspool/manager/ntp.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="OSCP" File="/bspool/manager/oscp.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="SSL" File="/bspool/manager/ssl.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="StdErr" File="/bspool/manager/stderr.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="StdOut" File="/bspool/manager/stdout.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="HTTP" File="/bspool/manager/http.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Notice" File="/bspool/manager/notice.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Weird" File="/bspool/manager/weird.log")

# define input files forwardSysLogs logs to send to the rsyslog server
# and apply ruleset "forwardSysLogs" 
# in /alog

input(type="imfile" ruleset="forwardSysLogs" Tag="ACL-Alarm" File="/alog/acl-alarm.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="ACL-Audit" File="/alog/acl-audit.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="DNS-Masq" File="/alog/dnsmasq-acl.log")

# define input files forwardSysLogs logs to send to the rsyslog server
# and apply ruleset "forwardSysLogs" 
# in /alog/firewalla

input(type="imfile" ruleset="forwardSysLogs" Tag="FireApi" File="/alog/firewalla/FireApi.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireKick" File="/alog/firewalla/FireKick.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireMain" File="/alog/firewalla/FireMain.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireMon" File="/alog/firewalla/FireMon.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireRouter" File="/alog/firewalla/FireRouter.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Trace" File="/alog/firewalla/Trace.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="CleanLog" File="/alog/firewalla/clean_log.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Firelog" File="/alog/firewalla/firelog.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Node" File="/alog/firewalla/node.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="SyncTime" File="/alog/firewalla/sync_time.log")

# Sending all other Syslog logs to Server (Synology) 
# @@IP is for TCP
# @IP is for UDP
*.* @@172.16.2.20:514

#########################
Modifying the config
########################

Be sure to change the following attributes in the new config file before pasting via VI into the syslog conf file:

        target="172.16.2.20"  # set your Synology Syslog Server NAS IP
        port="514"  # Specify port number
        protocol="tcp"  # specify protocol UDP or TCP

AND

# Sending all other Syslog logs to Server (Synology) 
# @@IP is for TCP
# @IP is for UDP
*.* @@172.16.2.20:514

#########################################
Setting up Syslog on Firewalla to send to Synology
#########################################

To collect the all of the important Firewalla modify the existing syslog conf file or create a new one by doing the following:

  • Go to the following directory by running the following command:
    cd /etc/rsyslog.d

  • Run ls -lar to find the 09-externalserver.conf file.

  • If the conf file doesn't exist then you'll need to create the the Syslog conf file (skip this step if 09-externalserver.conf does exist):
    sudo touch /etc/rsyslog.d/09-externalserver.conf
    sudo vi /etc/rsyslog.d/09-externalserver.conf

  • If the 09-externalserver.conf file exists because you already ran the script on this Github then you'll need to delete the script that created it (this step is extremely important or your new config will be overwritten by the script).
    sudo rm -rf /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh

  • Since deleting the script, we can keep the file name in place if it already exists but we want to erase everything from that file by running the following command (if you created the conf file using the touch command then skip this step since it will already be empty):
    sudo sed -i d 09-externalserver.conf

  • Next we need to open the file in order to paste our new configs by running the following command:
    sudo vi 09-externalserver.conf

  • Press the letter i on your keyboard for insert

  • Copy the configs and paste them into the file by right clicking (this is how you paste using VIM)

  • Once the configs are copied then press escape then type :wq! on your kyboard and hit enter

  • Now run the following command to restart the syslog engine:
    sudo systemctl restart rsyslog

That's it! You should now be grabbing all of the important Firewalla logs!

######################
Synology Log Center Results
######################

Here is a screenshot of my Synology Log Center:

image

@mjaestewart
Copy link

Hello, is it possible to send /log/blog to the remote syslog server? I would like to see the src and dst IP from the /log/blog/current/conn.log

Just made a solution for exactly what it is you're looking to do:

https://gist.github.com/mbierman/f3d184b65e0f4de6fa75a4a5d5145426?permalink_comment_id=4787139#gistcomment-4787139

@mjaestewart
Copy link

mjaestewart commented Dec 9, 2023

Also, since the packet headers are recived by the syslog server as localhost (which really bothers me). Syslog will always send the events based on the entry of the hosts file. So add a line above the default local addresses with your devices assigned IP address tabbed with a hostname of your liking (ie firewalla).

  • cd /etc and sudo vi hosts to create the following entry:
  • Press i for insert
  • Create the following entry with your own locally assigned static IP address:
172.16.2.1      firewalla     firewalla.lan
127.0.0.1   localhost Firewalla
  • Press esc after inserting your info then type :wq! and hit enter
  • Now run the following command to restart the syslog engine:
    sudo systemctl restart rsyslog

Now you will see logs come in as firewalla instead of localhost!

Results:

image

@RayBishopTN
Copy link

Hello @mjaestewart
I appreciate this very much, but for some reason, it is not working for me. When doing a tcpdump I see no traffic using port 514 or if I do it by host I do not see it making a connection to my NAS.

@mjaestewart
Copy link

mjaestewart commented Dec 11, 2023

Hello @mjaestewart I appreciate this very much, but for some reason, it is not working for me. When doing a tcpdump I see no traffic using port 514 or if I do it by host I do not see it making a connection to my NAS.

##########
Update
##########

Did you setup the syslog inputs on the synology in log center?

If not go to Log Center > Log Receiving > Create > Give your connection a name, then specify whether you want to use TCP or UDP on port 514. BSD format is fine as well.

On thing that I found on my end as well is that the script was never removed /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh and will overwrite the sylog file that was created.

So we have to remove that script by doing the following:

  • sudo rm -rf /home/pi/.firewalla/config/post_main.d/addremotesyslog.sh

  • Then change the IP AND Protocol information in the config file I posted to reflect your environment Conf Here

        target="172.16.2.20"  # set your Synology Syslog Server NAS IP
        port="514"  # Specify port number
        protocol="tcp"  # specify protocol UDP or TCP
  • Recreate the the Syslog conf file:
    sudo touch /etc/rsyslog.d/09-externalserver.conf
    sudo vi /etc/rsyslog.d/09-externalserver.conf

  • Press the letter i on your keyboard for insert

  • Copy the configs Conf Here and paste them into the file by right clicking (this is how you paste using VIM)

  • Once the configs are copied then press escape then type :wq! on your kyboard and hit enter

  • Now run the following command to restart the syslog engine:
    sudo systemctl restart rsyslog

@bn1980
Copy link

bn1980 commented Jan 11, 2024

Can this be used to get Firewalla gold networking blocks and IDS in to Wazuh?

@mbierman
Copy link
Author

@bn1980 I assume so. It looks like Wazuh supports syslog input.

@tsqrd
Copy link

tsqrd commented Jan 30, 2024

@mjaestewart This has worked great, but inevitably the zeek events stop streaming because I think zeek rotates those logs and then rsyslog doesn't pick up the change and is reading from the wrong inode. Have you seen this same behavior? How were you able to handle it? I don't see a logrotate conf for the zeek logs, so I assume it's the builtin zeek functionality for rotation.

@mjaestewart
Copy link

@mjaestewart This has worked great, but inevitably the zeek events stop streaming because I think zeek rotates those logs and then rsyslog doesn't pick up the change and is reading from the wrong inode. Have you seen this same behavior? How were you able to handle it? I don't see a logrotate conf for the zeek logs, so I assume it's the builtin zeek functionality for rotation.

I’ll put together a solution tomorrow and post it :-) Yes, I also see the same behavior.

@mbierman
Copy link
Author

@mjaestewart if you find a solution I'd love to test and incorporate it.

@mjaestewart
Copy link

mjaestewart commented Jan 30, 2024

@mbierman and @tsqrd

Here is my updated solution. I've tested all day, and so far so good. @mbierman I reused what you had already done, and built on that 👍

Script

#!/bin/bash
# v 2.1.0

script_location="/home/pi/.firewalla/config/post_main.d/" # script location
script="firewalla_rsyslog.sh" # script used to install firewalla syslog
cron_cmd="0 * * * * cd $script_location && sudo ./$script -c"
syslog="/etc/rsyslog.d/09-externalserver.conf" # rsyslog location
server="172.16.2.20" # Change the server to the IP of your syslog server.
port="514" # port used for forwarding logs to destination
protocol="tcp" #use tcp or udp
other_protocol="@@" # use @@ for TCP and @ for UDP
valid=$(grep "$server:$port" $syslog 2>/dev/null)



### Creating the syslog file

create() {
	sudo touch $syslog
	sudo cat > $syslog <<EOF
	
\$LocalHostName Firewalla
# deifne global workDirectory for saving the state file of log messages.
global(workDirectory="/var/spool/rsyslog")

# enable the Rsyslog imfile module processing text files or logs.
module(load="imfile" PollingInterval="10")


# define template for StandardSyslogFormat for processing log messages.
# that will be forwarded to rsyslog server
template(
    name="StandardSyslogFormat"
    type="string"
    string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
    )


# define ruleset "forwardSysLogs" with action object to send logs to rsyslog server
# define the queue
ruleset(name="forwardSysLogs") {
    action(
        type="omfwd"
        target="$server"  # set your Synology Syslog Server NAS IP
        port="$port"  # Specify port number
        protocol="$protocol"  # specify protocol UDP or TCP
        template="StandardSyslogFormat"  # specifies the template to use above

        queue.SpoolDirectory="/var/spool/rsyslog"
        queue.FileName="remote"
        queue.MaxDiskSpace="1g"
        queue.SaveOnShutdown="on"
        queue.Type="LinkedList"
        ResendLastMSGOnReconnect="on"
        )
        stop
}

# define input files forwardSysLogs logs to send to the rsyslog server
# and apply ruleset "forwardSysLogs" 
# in /bspool/manager

input(type="imfile" ruleset="forwardSysLogs" Tag="ConnLog" File="/bspool/manager/conn.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="ConnLongLog" File="/bspool/manager/conn_long.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="DNS" File="/bspool/manager/dns.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Files" File="/bspool/manager/files.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="HeartBeat" File="/bspool/manager/heartbeat.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="NTP" File="/bspool/manager/ntp.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="OSCP" File="/bspool/manager/oscp.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="SSL" File="/bspool/manager/ssl.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="StdErr" File="/bspool/manager/stderr.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="StdOut" File="/bspool/manager/stdout.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="HTTP" File="/bspool/manager/http.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Notice" File="/bspool/manager/notice.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Weird" File="/bspool/manager/weird.log")

# define input files forwardSysLogs logs to send to the rsyslog server
# and apply ruleset "forwardSysLogs" 
# in /alog

input(type="imfile" ruleset="forwardSysLogs" Tag="ACL-Alarm" File="/alog/acl-alarm.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="ACL-Audit" File="/alog/acl-audit.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="DNS-Masq" File="/alog/dnsmasq-acl.log")

# define input files forwardSysLogs logs to send to the rsyslog server
# and apply ruleset "forwardSysLogs" 
# in /alog/firewalla

input(type="imfile" ruleset="forwardSysLogs" Tag="FireApi" File="/alog/firewalla/FireApi.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireKick" File="/alog/firewalla/FireKick.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireMain" File="/alog/firewalla/FireMain.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireMon" File="/alog/firewalla/FireMon.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="FireRouter" File="/alog/firewalla/FireRouter.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Trace" File="/alog/firewalla/Trace.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="CleanLog" File="/alog/firewalla/clean_log.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Firelog" File="/alog/firewalla/firelog.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="Node" File="/alog/firewalla/node.log")
input(type="imfile" ruleset="forwardSysLogs" Tag="SyncTime" File="/alog/firewalla/sync_time.log")

# Sending all other Syslog logs to Server (Synology) 
# @@IP is for TCP
# @IP is for UDP
*.* $other_protocol$server:$port
EOF

    echo "Restarting rsyslog..."
    sudo systemctl restart rsyslog
    echo "remote syslog added"
    echo "adding cron job for reliability"
    (crontab -u pi -l 2>/dev/null; echo "$cron_cmd") | crontab -u pi -
    sudo systemctl restart cron
    exit
}

cleanup() {
	sudo rm -f $syslog
	sudo systemctl restart rsyslog
	(crontab -u pi -l | grep -vF "$cron_cmd" | crontab -u pi -)	
}

if [ -f "$syslog" ] ; then
	if [ -n "$valid" ] ; then
	echo "remote syslog already in place with $server:$port specified"
	case $1 in
		-c)
		echo -e "\nrecreating syslog configuration..." 
		cleanup
		create
		;;
  		-r|-restart|-force|-f)
                echo "Restarting rsyslog..."
                sudo systemctl restart rsyslog
		exit
		;;
		-u|-update)
			read -p "Are you sure you want to remove the syslog forwarder? type 'y' " -n 1 -r
			echo
			if [[ $REPLY =~ ^[Yy]$ ]] ; then
				ls $syslog 2>/dev/null && cleanup || echo -e "\n\nNo log found.\n"
			fi
			exit
		;;
		-h)
		echo -e "You can use:\n     - \`$0 -c\` recreate forwarding\n     - \`$0 -r\` restart the syslog service\
		\n     - \`$0 -u\` uninstall the settings to send to the remote syslog server\n\n"
    		exit
		;;
	esac
	else
		echo "The server is not configured correctly. On it." 
		cleanup
		create
	fi
else
	echo "There was no syslog forwarder in place."
	create
fi

Fixes

  • Hostname is now set to Firewalla
  • Cron is now used to ensure persistent sending of all FW logs
  • Implementation is now completely automated via script

Setting up the Directory

To send logs to a remote syslog server using UDP, do the following:

  1. ssh to the Firewalla box.
  2. Copy the script above.
  3. If /home/pi/.firewalla/config/post_main.d/ doesn’t exist, create it first.
    sudo mkdir /home/pi/.firewalla/config/post_main.d/
  4. Next, create the file:
    sudo vi /home/pi/.firewalla/config/post_main.d/firewalla_rsyslog.sh

Modifying the Variables in the Script and Executing

  1. Edit the following variables in the script for your specific environment: 
    1. server to the IP address of your syslog server.
    2. port to the correct port being used for rsyslog
    3. protocol to specify TCP or UDP
    4. other_protocol uses a single @ for UDP and a double @@ for TCP
  2. Paste this script into firewalla_rsyslog.sh. This is going to create rsyslog configs and the cron job that runs to ensure that the syslog setting remains in place, even if there's a firewalla update that wipes out the settings in the future.
  3. Save the file :wq!
  4. Give the script execute permissions.
    sudo chmod +x /home/pi/.firewalla/config/post_main.d/firewalla_rsyslog.sh
  5. Execute the script.
    sudo /home/pi/.firewalla/config/post_main.d/firewalla_rsyslog.sh -c creates the file and restarts syslog

Additional Arguments

  • /home/pi/.firewalla/config/post_main.d/firewalla_rsyslog.sh -r restarts syslog
  • /home/pi/.firewalla/config/post_main.d/firewalla_rsyslog.sh -u uninstalls the forwarder and restarts syslog.

@mjaestewart
Copy link

Screenshot 2024-01-30 151632

@tsqrd
Copy link

tsqrd commented Feb 23, 2024

@mjaestewart Do you end up with an endless supply of imfile state files in /var/spool/rsyslog? I ended up adding a cronjob to delete files older than 5 minutes in that directory because otherwise it just fills up indefinitely. I'm assuming it has something to do with zeek truncating/rotating the log files because I also end up with these messages from rsyslogd in /var/log/syslog: imfile: internal error? inotify provided watch descriptor 3745 which we could not find in our tables.

@tsqrd
Copy link

tsqrd commented Feb 28, 2024

You may also want to consider this post about persisting cron through reboots/restarts: https://help.firewalla.com/hc/en-us/articles/360054056754-Customized-Scripting

I notice the cronjob disappeared after a reload so I added it to the location described by that article.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment