Skip to content

Instantly share code, notes, and snippets.

@adotagarwal
Forked from tblong/rpi-network-watchdog.sh
Last active January 8, 2024 15:39
Show Gist options
  • Save adotagarwal/b6e4bb09e051638271fa3aff910f8eb4 to your computer and use it in GitHub Desktop.
Save adotagarwal/b6e4bb09e051638271fa3aff910f8eb4 to your computer and use it in GitHub Desktop.
[Daemon] Checks a pingable source and attempts to recover WiFi when connection has been interrupted by having the kernel reload the SDIO drivers
#!/bin/bash
#
# Original Source Attribution:
# Location: https://gist.github.com/SandroMachado/87e591fc42f368636b251b566485ae46
# Author: Sandro Machado
# Date of retrieval: 2020-01-25
#
# Script Summary:
# This script checks a pingable source and when the failure
# limit is reached the host is restarted. No files are written
# to with this script.
#
# This script is intended to be ran via cron with the @reboot
# (see man 5 crontab) option so that it runs once at startup and
# continues execution in an infinite loop until a reboot
# condition is reached.
#
# Modifications the original source are made under the following license:
#
# MIT License
#
# Copyright (c) 2020 Austin Agarwal
# Copyright (c) 2019 Tyler B. Long
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
####################
# Configurations
####################
# The interval [sec] to check the pingable destination for connectivity.
check_interval=30
# The public or local resource to ping.
ping_destination=google.com
# The number of ping packets to send.
ping_count=3
# The consecutive max ping failure count. When reached a reboot is invoked.
max_ping_failures=5
####################
# Main
####################
failure_count=0
while true
do
# test the pingable resource
ping -c $ping_count $ping_destination > /dev/null 2>&1
# check exit status of ping command
if [[ $? -eq 0 ]]; then
failure_count=0
echo 'All good as of:' $(date)
else
failure_count=$((failure_count+1))
echo 'Network failure at:' $(date)
if ((failure_count >= max_ping_failures)); then
echo 'Recovering network stack at:' $(date)
sudo ifdown wlan0
sleep 5
sudo modprobe -r brcmfmac
sleep 1
sudo modprobe brcmfmac
sleep 5
sudo ifup wlan0
failure_count=0
sleep 5
fi
fi
sleep $check_interval
done
@adotagarwal
Copy link
Author

adotagarwal commented Jan 31, 2020

Intro:
This script will: take down wlan0, then unload and reload the kernel module for the SDIO hardware, which includes the wifi and bluetooth stack, and bring back wlan0.

Problem:
I've noticed the wireless connections (wifi, bluetooth) on my Raspberry Pi 4 will degrade randomly -- sometimes a few hours after a reboot and sometimes several days after a reboot. The wifi will drop all together, it may seem as if its connected but am unable to ping anything. The bluetooth range drops dramatically when this happens as well. Rebooting fixes the issue, but this seems unacceptable for a Linux device. I'd rather have windows if this was the only solution.

Thankfully, I've discovered the following resolution. This runs as a daemon, meaning it is designed to be started only once

Resources:

-When the wifi drops, I receive errors in various logs such as: [76229.046423] brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted

@hovancik
Copy link

Hey @adotagarwal, where should I be able to see outputs of echo ?

@adotagarwal
Copy link
Author

adotagarwal commented Sep 23, 2020 via email

@jeff-porticos
Copy link

Why do you break after bringing wlan0 back up? Doesn't this limit the script to a single shot recovery?

@adotagarwal
Copy link
Author

Why do you break after bringing wlan0 back up? Doesn't this limit the script to a single shot recovery?

This is a good point -- to be honest I use the other version of this gist i have here on a cronjob every 5 minutes.

The break is a ruminant of the fork, as the original reboots the whole machine and presumably the script starts again on boot.

@sebekhtc
Copy link

sebekhtc commented Jan 8, 2024

having unresponsive RPi I found this script (thanks!). As another idea, I would like to reboot the RPI (or reload kernel like here) on demand - as I use Zigbee and Z-wave sticks, I can test USB ports on their presence and when both unplugged (by me) reboot the RPi. Quickly made the below run in cron every 5 minutes

#!/bin/bash
result=$(ls /dev/serial/by-id/*usb-0658_0200-if00)
if [ "$result" == "/dev/serial/by-id/usb-0658_0200-if00" ]
then
exit
fi

result=$(ls /dev/serial/by-id/usb-Silicon_Labs_Sonoff_Zigbee_3.0_USB_Dongle_Plus_0001-if00-port0)
f [ "$result" == "/dev/serial/by-id/usb-Silicon_Labs_Sonoff_Zigbee_3.0_USB_Dongle_Plus_0001-if00-port0" ]
then
exit
fi

echo "Zwave and Zigbee dongles not found, rebooting RPi..."
sudo service domoticz.sh stop
sudo systemctl reboot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment