Skip to content

Instantly share code, notes, and snippets.

@teffalump
Last active January 4, 2023 21:17
Star You must be signed in to star a gist
Save teffalump/7227752 to your computer and use it in GitHub Desktop.
OpenWRT adblock implementation

Others have recently developed packages for this same functionality, and done it better than anything I could do. Use the packages instead of this script:

Description

In its basic usage, this script will modify the router such that blocked addresses are null routed and unreachable. Since the address blocklist is full of advertising, malware, and tracking servers, this setup is generally a good thing. In addition, the router will update the blocklist weekly. However, the blocking is leaky, so do not expect everything to be blocked.

Setup

The script must be copied to an OpenWRT router (gargoyle firmware works fine, too).

For example, if the router is located at 192.168.1.1:

# scp adblock.sh root@192.168.1.1:/etc/adblock.sh

Make the script executable:

# chmod +x /etc/adblock.sh

Basic usage

If you are running the script for the first time:

# sh /etc/adblock.sh -f

There should be status updates in the output, but there should be no errors. If these commands complete without errors, the adblocking is active. You can test it by looking up, say, google analytics.

Whitelists and blacklists

The script supports defining whitelisted urls. That is, urls that will be filtered out of the downloaded blocklists. To whitelist urls, place them (one per line) in /etc/white.list.

Similarly, the script supports defining blacklisted urls - urls that will be added to the downloaded blocklists. To blacklist urls, place them (one per line) in /etc/black.list.

NOTE: The whitelist support is pretty stupid, so don't expect smart filtering (e.g., domain extrapolation). I've found it tedious, but worthwhile, to find the offending url in /etc/block.hosts and copy it to /etc/white.list.

Advanced usage

Toggle on and off

To toggle the blocking on and off, run the script with the -t switch:

# sh /etc/adblock.sh -t

Note: This does not delete the blocklist, whitelist, or blacklist.

Manually update blocklist

To manually update the blocklist, run the script without switches:

# sh /etc/adblock.sh

Full reinstall

To reinstall the current implementation:

# sh /etc/adblock.sh -r

Configuration

The config section of the script has some variables that alter the behaviour of the script.

For example, if you change:

ONLY_WIRELESS="N"

to

ONLY_WIRELESS="Y"

Then only the wireless interface of the router will filter the blocklist.

To change the configuration of an already active installation, I would toggle the adblocking off first, change the script, then toggle it back on. That is,

# sh /etc/adblock.sh -t # turn off
...modify variables...
# sh /etc/adblock.sh -t # turn on

However, if you change certain variables, you must re-update the blocklist because the redirection values will have changed. Other variables require a full restart because they must install or verify dependencies.

Configurable variables:

  • ONLY_WIRELESS (Y/N): Only filter on wireless interface
  • EXEMPT# (Y/N): Exempt ip range from filtering (between START_ RANGE and END_RANGE)
  • IPV6*: (Y/N): Add IPv6 support
  • SSL# (Y/N): Install wget with ssl support (only needed for ssl websites)
  • TRANS#: (Y/N): Modify router web server to server transparent pixel responses for blocked websites
  • ENDPOINT_IP4/IP6*: Define the IP to return for blocked hostnames (IPv4 and IPv6)
  • CRON: The cron line to put in the crontab

*: Requires blocklist update.

: Requires full reinstall

#!/bin/sh
#Put in /etc/adblock.sh
#Block ads, malware, etc.
#### CONFIG SECTION ####
# Only block wireless ads? Y/N
ONLY_WIRELESS="N"
# IPv6 support? Y/N
IPV6="N"
# Need SSL websites?
SSL="N"
# Try to transparently serve pixel response?
# If enabled, understand the consequences and mechanics of this setup
TRANS="N"
# Exempt an ip range
EXEMPT="N"
START_RANGE="192.168.1.0"
END_RANGE="192.168.1.255"
# Redirect endpoint
ENDPOINT_IP4="0.0.0.0"
ENDPOINT_IP6="::"
#Change the cron command to what is comfortable, or leave as is
CRON="0 4 * * 0,3 sh /etc/adblock.sh"
#### END CONFIG ####
#### FUNCTIONS ####
cleanup()
{
#Delete files used to build list to free up the limited space
echo 'Cleaning up...'
rm -f /tmp/block.build.list
rm -f /tmp/block.build.before
}
install_dependencies()
{
#Need iptables-mod-nat-extra installed
if opkg list-installed | grep -q iptables-mod-nat-extra
then
echo 'iptables-mod-nat-extra is installed!'
else
echo 'Updating package list...'
opkg update > /dev/null
echo 'Installing iptables-mod-nat-extra...'
opkg install iptables-mod-nat-extra > /dev/null
fi
#Need iptable-mod-iprange for exemption
if [ "$EXEMPT" = "Y" ]
then
if opkg list-installed | grep -q iptables-mod-iprange
then
echo 'iptables-mod-iprange installed'
else
echo 'Updating package list...'
opkg update > /dev/null
echo 'Installing iptables-mod-iprange...'
opkg install iptables-mod-iprange > /dev/null
fi
fi
#Need wget for https websites
if [ "$SSL" = "Y" ]
then
if opkg list-installed wget | grep -q wget
then
if wget --version | grep -q +ssl
then
echo 'wget (with ssl) found'
else
# wget without ssl, need to reinstall full wget
opkg update > /dev/null
opkg install wget --force-reinstall > /dev/null
fi
else
echo 'Updating package list...'
opkg update > /dev/null
echo 'Installing wget (with ssl)...'
opkg install wget > /dev/null
fi
fi
}
add_config()
{
if [ "$ONLY_WIRELESS" = "Y" ]
then
echo 'Wireless only blocking!'
if [ "$EXEMPT" = "Y" ]
then
echo 'Exempting some ips...'
FW1="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -i wlan+ -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -i wlan+ -p udp --dport 53 -j REDIRECT --to-ports 53"
else
FW1="iptables -t nat -I PREROUTING -i wlan+ -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -i wlan+ -p udp --dport 53 -j REDIRECT --to-ports 53"
fi
else
if [ "$EXEMPT" = "Y" ]
then
echo "Exempting some ips..."
FW1="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -p udp --dport 53 -j REDIRECT --to-ports 53"
else
FW1="iptables -t nat -I PREROUTING -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -p udp --dport 53 -j REDIRECT --to-ports 53"
fi
fi
echo 'Updating config...'
#Update DHCP config
uci add_list dhcp.@dnsmasq[0].addnhosts=/etc/block.hosts > /dev/null 2>&1 && uci commit
#Add to crontab
echo "$CRON" >> /etc/crontabs/root
#Update dnsmasq config for Tor
TOR=`uci get tor.global.enabled 2> /dev/null`
if [ "$TOR" == "1" ]
then
TORPORT=`uci get tor.client.dns_port`
TORIP="127.0.0.1:$TORPORT"
uci set dhcp.@dnsmasq[0].noresolv='1' > /dev/null &2>1 && uci commit
uci add_list dhcp.@dnsmasq[0].server="$TORIP" > /dev/null &2>1 && uci commit
fi
# Add firewall rules
echo "$FW1" >> /etc/firewall.user
echo "$FW2" >> /etc/firewall.user
# Provide hint if localservice is 1
LS=`uci get dhcp.@dnsmasq[0].localservice 2> /dev/null`
if [ "$LS" == "1" ]
then
echo "HINT: localservice is set to 1"
echo " Adblocking (and router DNS) over a VPN may not work"
echo " To allow VPN router DNS, manually set localservice to 0"
fi
# Determining uhttpd/httpd_gargoyle for transparent pixel support
if [ "$TRANS" = "Y" ]
then
if [ ! -e "/www/1.gif" ]
then
/usr/bin/wget -O /www/1.gif http://upload.wikimedia.org/wikipedia/commons/c/ce/Transparent.gif > /dev/null
fi
if [ -s "/usr/sbin/uhttpd" ]
then
#The default is none, so I don't want to check for it, so just write it
echo "uhttpd found..."
echo "updating server error page to return transparent pixel..."
uci set uhttpd.main.error_page="/1.gif" && uci commit
elif [ -s "/usr/sbin/httpd_gargoyle" ]
then
# Write without testing
echo "httpd_gargoyle found..."
echo "updating server error page to return transparent pixel..."
uci set httpd_gargoyle.server.page_not_found_file="1.gif" && uci commit
else
echo "Cannot find supported web server..."
fi
fi
}
update_blocklist()
{
#Delete the old block.hosts to make room for the updates
rm -f /etc/block.hosts
# Correct endpoint for transparent pixel response
if [ "$TRANS" = "Y" ] && [ -e "/www/1.gif" ] && ([ -s "/usr/sbin/uhttpd" ] || [ -s "/usr/sbin/httpd_gargoyle" ])
then
ENDPOINT_IP4=$(uci get network.lan.ipaddr)
if [ "$IPV6" = "Y" ]
then
ENDPOINT_IP6=$(uci get network.lan6.ipaddr)
fi
fi
echo 'Downloading hosts lists...'
#Download and process the files needed to make the lists (enable/add more, if you want)
wget -qO- http://www.mvps.org/winhelp2002/hosts.txt| awk -v r="$ENDPOINT_IP4" '{sub(/^0.0.0.0/, r)} $0 ~ "^"r' > /tmp/block.build.list
wget -qO- "http://adaway.org/hosts.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- http://www.malwaredomainlist.com/hostslist/hosts.txt|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- "http://hosts-file.net/.\ad_servers.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#Add black list, if non-empty
if [ -s "/etc/black.list" ]
then
echo 'Adding blacklist...'
awk -v r="$ENDPOINT_IP4" '/^[^#]/ { print r,$1 }' /etc/black.list >> /tmp/block.build.list
fi
echo 'Sorting lists...'
#Sort the download/black lists
awk '{sub(/\r$/,"");print $1,$2}' /tmp/block.build.list|sort -u > /tmp/block.build.before
#Filter (if applicable)
if [ -s "/etc/white.list" ]
then
#Filter the blacklist, supressing whitelist matches
# This is relatively slow =-(
echo 'Filtering white list...'
egrep -v "^[[:space:]]*$" /etc/white.list | awk '/^[^#]/ {sub(/\r$/,"");print $1}' | grep -vf - /tmp/block.build.before > /etc/block.hosts
else
cat /tmp/block.build.before > /etc/block.hosts
fi
if [ "$IPV6" = "Y" ]
then
safe_pattern=$(printf '%s\n' "$ENDPOINT_IP4" | sed 's/[[\.*^$(){}?+|/]/\\&/g')
safe_addition=$(printf '%s\n' "$ENDPOINT_IP6" | sed 's/[\&/]/\\&/g')
echo 'Adding ipv6 support...'
sed -i -re "s/^(${safe_pattern}) (.*)$/\1 \2\n${safe_addition} \2/g" /etc/block.hosts
fi
}
restart_firewall()
{
echo 'Restarting firewall...'
if [ -s "/usr/lib/gargoyle/restart_firewall.sh" ]
then
/usr/lib/gargoyle/restart_firewall.sh > /dev/null 2>&1
else
/etc/init.d/firewall restart > /dev/null 2>&1
fi
}
restart_dnsmasq()
{
if [ "$1" -eq "0" ]
then
echo 'Re-reading blocklist'
killall -HUP dnsmasq
else
echo 'Restarting dnsmasq...'
/etc/init.d/dnsmasq restart
fi
}
restart_http()
{
if [ -s "/usr/sbin/uhttpd" ]
then
echo 'Restarting uhttpd...'
/etc/init.d/uhttpd restart
elif [ -s "/usr/sbin/httpd_gargoyle" ]
then
echo 'Restarting httpd_gargoyle...'
/etc/init.d/httpd_gargoyle restart
fi
}
restart_cron()
{
echo 'Restarting cron...'
/etc/init.d/cron restart > /dev/null 2>&1
}
remove_config()
{
echo 'Reverting config...'
# Remove addnhosts
uci del_list dhcp.@dnsmasq[0].addnhosts=/etc/block.hosts > /dev/null 2>&1 && uci commit
# Remove cron entry
sed -i '/adblock/d' /etc/crontabs/root
# Remove firewall rules
sed -i '/--to-ports 53/d' /etc/firewall.user
# Remove Tor workarounds
uci del_list dhcp.@dnsmasq[0].server > /dev/null 2>&1 && uci commit
uci set dhcp.@dnsmasq[0].noresolv='0' > /dev/null 2>&1 && uci commit
# Remove proxying
uci delete uhttpd.main.error_page > /dev/null 2>&1 && uci commit
uci set httpd_gargoyle.server.page_not_found_file="login.sh" > /dev/null 2>&1 && uci commit
}
toggle()
{
# Check for cron as test for on/off
if grep -q "adblock" /etc/crontabs/root
then
# Turn off
echo 'Turning off!'
remove_config
else
# Turn on
echo 'Turning on!'
add_config
fi
# Restart services
restart_firewall
restart_dnsmasq 1
restart_http
restart_cron
}
#### END FUNCTIONS ####
### Options parsing ####
case "$1" in
# Toggle on/off
"-t")
toggle
;;
#First time run
"-f")
install_dependencies
add_config
update_blocklist
restart_firewall
restart_dnsmasq 1
restart_http
restart_cron
cleanup
;;
#Reinstall
"-r")
remove_config
install_dependencies
add_config
update_blocklist
restart_firewall
restart_dnsmasq 1
restart_http
restart_cron
cleanup
;;
#Default updates blocklist only
*)
update_blocklist
restart_dnsmasq 0
cleanup
;;
esac
#### END OPTIONS ####
exit 0
#/etc/black.list
#add some server that the list doesn't block
example1.block.com
#/etc/sysupgrade.conf
#This file is a list of files that should be preserved through upgrades
#OPTIONAL!!!!!
/etc/passwd
/etc/shadow
...
...
/etc/adblock.sh #ADD THIS LINE
/etc/white.list #AND THIS ONE
/etc/block.hosts #AND THIS ONE
/etc/black.list #AND THIS ONE
#/etc/white.list
#Add whitelisted addresses, when appropriate, etc.
a248.e.akamai.net
@teffalump
Copy link
Author

Just added the hint, although maybe I should stress more caution. Lol.

@frontalot
Copy link

I run a persistent OpenVPN connection on my router and ad blocking wasn't working at first. Removing the localservice 1 entry from my dnsmasq config worked.

Is there any way to make the ad blocking work without removing the localservice entry? I don't want to open myself up to security issues...

@frontalot
Copy link

Also, when turning on your script, I get a warning line:

Turning on!
Updating config...
uci: Entry not found
Restarting firewall...
Restarting dnsmasq...
Restarting uhttpd...
Restarting cron...

Is the uci warning normal or expected?

@MichiMunich
Copy link

@frontalot
Probably it is because of the command "TOR=uci get tor.global.enabled"
If you don't use TOR you get "uci: Entry not found" as result. So it's not a problem.

@bonanza123
Copy link

when I enable the 'serving with transparent pixel' option then it actually redirects to my openWrt router right? But when it does so, doesn't it interfere with the LUCI login page also listening on port 80 and 443?

@teffalump
Copy link
Author

@frontalot: From what I understand, as long as you take normal precautions, like not listening on external DNS port, then it shouldn't be a problem. But, yeah, I'm not too knowledgeable about this stuff. =-/

@teffalump
Copy link
Author

Pushed change for that Tor check. Shouldn't have that error now. =-)

@teffalump
Copy link
Author

@bonanza123: You are right. =-) Although, what happens is that the redirection has the client request a page from the router, say /advertpage.jsp, and the router cannot find it, so it serves the page_error response (which we've overriden). Consequently, there shouldn't be any problems with the normal operation of the Openwrt web server functionality. However, I have seen some issues. Generally, it's somewhat inelegant and creates subtle issues, but works most of the time. =-)

@dartraiden
Copy link

wget -qO- http://www.mvps.org/winhelp2002/hosts.txt| awk -v r="$ENDPOINT_IP4" '{sub(/^0.0.0.0/, r)} $0 ~ "^"r' > /tmp/block.build.list
wget -qO- "http://adaway.org/hosts.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- http://www.malwaredomainlist.com/hostslist/hosts.txt|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- "http://hosts-file.net/.\ad_servers.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list

Thanks for awesome script! Why links 2 and 4 have "", but 1 and 3 are not? It is important?

@teffalump
Copy link
Author

@dartraiden: Link 2 could be unquoted (I believe), but link 4 has a weird, literal backslash that must be kept uninterpreted. Although, it's disabled by default. =-)

@uneed
Copy link

uneed commented Oct 29, 2015

very useful,any plan support adblock plus's filters

@teffalump
Copy link
Author

@uneed: Nope. Wouldn't work for a variety of reasons: resource constraints, ssl security, etc. We've talked about this before... somewhere up in the comments. Hehe.

@frontalot
Copy link

I've noticed that some hosts are added to block.hosts as "0.0.0.0domainnamehere.com", with the space missing. Maybe your awk code needs to add a leading space if the hosts lists contains just domains?

@lantis1008
Copy link

But none of the included lists do not have leading ip's. And looking at my own list I don't observe this behaviour at all.

@teffalump
Copy link
Author

Yeah, like @lantis1008 said, I don't see that behavior @frontalot. There shouldn't be naked domains on the lists. Hmm...no matter, I'll keep an eye out for it. Thanks!

@n-land
Copy link

n-land commented Nov 20, 2015

@teffalump: I'd like to try this great script, what is the license? MIT? GPL?

@illitum
Copy link

illitum commented Nov 22, 2015

do u create menu position on openwrt to manage this script?

@teffalump
Copy link
Author

@n-land: Say GPL, and we'll go from there. Thanks! =-)

@illitum: No, I need to create the package to have easier management and options, like the menu position. But I've been somewhat lazy in that regard. =-)

@remlei
Copy link

remlei commented Dec 12, 2015

it would be cool if I can define a path where to store the blacklist.host file instead of storing it on my router /etc folder and make my router's flash goes bad more quickly.

I use a adblock on tomato but I can define to store the host files on my RAM, which is cool.

Yeah yeah, I know, I can edit the source if I want, anyway I hope you can work on that.

@b4hr4m
Copy link

b4hr4m commented Dec 17, 2015

Is this just work on incoming traffic from the WAN port? I'm using my phone to tethering via USB port and I still see the ads.

@teffalump
Copy link
Author

@b4hr4m: I'm having some some trouble understanding what the setup is. The phone is an access point? Or? Basically, if there are unencrypted DNS requests through the router, the ads should be blocked since the router will intercept them. Although there are some unusual cases.

@remlei: Sorry for the delay, but yeah, you have a good point. I should add the ability to define the white and blacklist location. I'll do it when I get back from vacation. In the meantime, yeah, just switch the locations manually. =-)

@Seroczynski
Copy link

Thanks for this great plugin. However, wireless only blocking isn't working for me. At first I configured the adblock to block all advertisement, which I later changed to wireless only. And while the script is clearly shows the wireless only option is enabled it still blocks everything on my wired network as well. I've tried reinstalling the script with the -r parameter but that didn't work.

I currently only have two lines in my iptables.

@remlei
Copy link

remlei commented Jan 25, 2016

im using mwan3 on openwrt, every time the blocklist updates, mwan3 breaks and need to restarted the service manually.

@norman25
Copy link

norman25 commented Feb 1, 2016

i thing the next step to do is creating a Web Interface For The Ad-blocking like the pi-hole web https://github.com/Anon135813/dnsblocker-webgui - http://jacobsalmela.com/a-web-interface-for-the-ad-blocking-pi-hole/ source https://pi-hole.net/

@lantis1008
Copy link

@norman25 my plugin already does this.

@norman25
Copy link

norman25 commented Feb 2, 2016

@lantis1008 wow i see it, but i can't use it.
my old router is not supported by gargoyle.

@teffalump
Copy link
Author

@norman25: I'm probably not going to add more to this script, the other plugins are way better now anyways - I would move to those. The openwrt package should work for your router, no? =-)

@teffalump
Copy link
Author

@remlei: Hmm... Yeah, no idea really -- updating is just rereading a new list, don't understand why it would break... I don't know mwan3 that well, although I perused the source. Lol. =-/

@fay27
Copy link

fay27 commented May 4, 2017

adaway now redirects to https. script needs to be updated

root@OpenWrt:/# wget http://adaway.org/hosts.txt
--2017-05-04 16:28:17-- http://adaway.org/hosts.txt
Resolving adaway.org... 2400:cb00:2048:1::6818:6959, 104.24.105.89, 104.24.104.89
Connecting to adaway.org|2400:cb00:2048:1::6818:6959|:80... failed: Permission denied.
Connecting to adaway.org|104.24.105.89|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://adaway.org/hosts.txt [following]
--2017-05-04 16:28:18-- https://adaway.org/hosts.txt
Connecting to adaway.org|104.24.105.89|:443... connected.
ERROR: cannot verify adaway.org's certificate, issued by 'CN=COMODO ECC Domain Validation Secure Server CA 2,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB':
Unable to locally verify the issuer's authority.
To connect to adaway.org insecurely, use `--no-check-certificate'.

@develsites
Copy link

Hi Guys

I have a problem with OpenVPN + this script. @dcavni It seems I have the same issue so, I'd like to know how you solved it

root@OpenWrt:~# uci get dhcp.@dnsmasq[0].localservice
0
root@OpenWrt:~# uci show | grep openvpn
...
openvpn.sample_server.server='10.8.0.0 255.255.255.0'

OpenVPN works without running this script

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment