Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
OpenWRT adblock implementation

Others have recently developed packages for this same functionality, and done it better than anything I could do. Use the packages instead of this script:

Description

In its basic usage, this script will modify the router such that blocked addresses are null routed and unreachable. Since the address blocklist is full of advertising, malware, and tracking servers, this setup is generally a good thing. In addition, the router will update the blocklist weekly. However, the blocking is leaky, so do not expect everything to be blocked.

Setup

The script must be copied to an OpenWRT router (gargoyle firmware works fine, too).

For example, if the router is located at 192.168.1.1:

# scp adblock.sh root@192.168.1.1:/etc/adblock.sh

Make the script executable:

# chmod +x /etc/adblock.sh

Basic usage

If you are running the script for the first time:

# sh /etc/adblock.sh -f

There should be status updates in the output, but there should be no errors. If these commands complete without errors, the adblocking is active. You can test it by looking up, say, google analytics.

Whitelists and blacklists

The script supports defining whitelisted urls. That is, urls that will be filtered out of the downloaded blocklists. To whitelist urls, place them (one per line) in /etc/white.list.

Similarly, the script supports defining blacklisted urls - urls that will be added to the downloaded blocklists. To blacklist urls, place them (one per line) in /etc/black.list.

NOTE: The whitelist support is pretty stupid, so don't expect smart filtering (e.g., domain extrapolation). I've found it tedious, but worthwhile, to find the offending url in /etc/block.hosts and copy it to /etc/white.list.

Advanced usage

Toggle on and off

To toggle the blocking on and off, run the script with the -t switch:

# sh /etc/adblock.sh -t

Note: This does not delete the blocklist, whitelist, or blacklist.

Manually update blocklist

To manually update the blocklist, run the script without switches:

# sh /etc/adblock.sh

Full reinstall

To reinstall the current implementation:

# sh /etc/adblock.sh -r

Configuration

The config section of the script has some variables that alter the behaviour of the script.

For example, if you change:

ONLY_WIRELESS="N"

to

ONLY_WIRELESS="Y"

Then only the wireless interface of the router will filter the blocklist.

To change the configuration of an already active installation, I would toggle the adblocking off first, change the script, then toggle it back on. That is,

# sh /etc/adblock.sh -t # turn off
...modify variables...
# sh /etc/adblock.sh -t # turn on

However, if you change certain variables, you must re-update the blocklist because the redirection values will have changed. Other variables require a full restart because they must install or verify dependencies.

Configurable variables:

  • ONLY_WIRELESS (Y/N): Only filter on wireless interface
  • EXEMPT# (Y/N): Exempt ip range from filtering (between START_ RANGE and END_RANGE)
  • IPV6*: (Y/N): Add IPv6 support
  • SSL# (Y/N): Install wget with ssl support (only needed for ssl websites)
  • TRANS#: (Y/N): Modify router web server to server transparent pixel responses for blocked websites
  • ENDPOINT_IP4/IP6*: Define the IP to return for blocked hostnames (IPv4 and IPv6)
  • CRON: The cron line to put in the crontab

*: Requires blocklist update.

: Requires full reinstall

#!/bin/sh
#Put in /etc/adblock.sh
#Block ads, malware, etc.
#### CONFIG SECTION ####
# Only block wireless ads? Y/N
ONLY_WIRELESS="N"
# IPv6 support? Y/N
IPV6="N"
# Need SSL websites?
SSL="N"
# Try to transparently serve pixel response?
# If enabled, understand the consequences and mechanics of this setup
TRANS="N"
# Exempt an ip range
EXEMPT="N"
START_RANGE="192.168.1.0"
END_RANGE="192.168.1.255"
# Redirect endpoint
ENDPOINT_IP4="0.0.0.0"
ENDPOINT_IP6="::"
#Change the cron command to what is comfortable, or leave as is
CRON="0 4 * * 0,3 sh /etc/adblock.sh"
#### END CONFIG ####
#### FUNCTIONS ####
cleanup()
{
#Delete files used to build list to free up the limited space
echo 'Cleaning up...'
rm -f /tmp/block.build.list
rm -f /tmp/block.build.before
}
install_dependencies()
{
#Need iptables-mod-nat-extra installed
if opkg list-installed | grep -q iptables-mod-nat-extra
then
echo 'iptables-mod-nat-extra is installed!'
else
echo 'Updating package list...'
opkg update > /dev/null
echo 'Installing iptables-mod-nat-extra...'
opkg install iptables-mod-nat-extra > /dev/null
fi
#Need iptable-mod-iprange for exemption
if [ "$EXEMPT" = "Y" ]
then
if opkg list-installed | grep -q iptables-mod-iprange
then
echo 'iptables-mod-iprange installed'
else
echo 'Updating package list...'
opkg update > /dev/null
echo 'Installing iptables-mod-iprange...'
opkg install iptables-mod-iprange > /dev/null
fi
fi
#Need wget for https websites
if [ "$SSL" = "Y" ]
then
if opkg list-installed wget | grep -q wget
then
if wget --version | grep -q +ssl
then
echo 'wget (with ssl) found'
else
# wget without ssl, need to reinstall full wget
opkg update > /dev/null
opkg install wget --force-reinstall > /dev/null
fi
else
echo 'Updating package list...'
opkg update > /dev/null
echo 'Installing wget (with ssl)...'
opkg install wget > /dev/null
fi
fi
}
add_config()
{
if [ "$ONLY_WIRELESS" = "Y" ]
then
echo 'Wireless only blocking!'
if [ "$EXEMPT" = "Y" ]
then
echo 'Exempting some ips...'
FW1="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -i wlan+ -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -i wlan+ -p udp --dport 53 -j REDIRECT --to-ports 53"
else
FW1="iptables -t nat -I PREROUTING -i wlan+ -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -i wlan+ -p udp --dport 53 -j REDIRECT --to-ports 53"
fi
else
if [ "$EXEMPT" = "Y" ]
then
echo "Exempting some ips..."
FW1="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -m iprange ! --src-range $START_RANGE-$END_RANGE -p udp --dport 53 -j REDIRECT --to-ports 53"
else
FW1="iptables -t nat -I PREROUTING -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING -p udp --dport 53 -j REDIRECT --to-ports 53"
fi
fi
echo 'Updating config...'
#Update DHCP config
uci add_list dhcp.@dnsmasq[0].addnhosts=/etc/block.hosts > /dev/null 2>&1 && uci commit
#Add to crontab
echo "$CRON" >> /etc/crontabs/root
#Update dnsmasq config for Tor
TOR=`uci get tor.global.enabled 2> /dev/null`
if [ "$TOR" == "1" ]
then
TORPORT=`uci get tor.client.dns_port`
TORIP="127.0.0.1:$TORPORT"
uci set dhcp.@dnsmasq[0].noresolv='1' > /dev/null &2>1 && uci commit
uci add_list dhcp.@dnsmasq[0].server="$TORIP" > /dev/null &2>1 && uci commit
fi
# Add firewall rules
echo "$FW1" >> /etc/firewall.user
echo "$FW2" >> /etc/firewall.user
# Provide hint if localservice is 1
LS=`uci get dhcp.@dnsmasq[0].localservice 2> /dev/null`
if [ "$LS" == "1" ]
then
echo "HINT: localservice is set to 1"
echo " Adblocking (and router DNS) over a VPN may not work"
echo " To allow VPN router DNS, manually set localservice to 0"
fi
# Determining uhttpd/httpd_gargoyle for transparent pixel support
if [ "$TRANS" = "Y" ]
then
if [ ! -e "/www/1.gif" ]
then
/usr/bin/wget -O /www/1.gif http://upload.wikimedia.org/wikipedia/commons/c/ce/Transparent.gif > /dev/null
fi
if [ -s "/usr/sbin/uhttpd" ]
then
#The default is none, so I don't want to check for it, so just write it
echo "uhttpd found..."
echo "updating server error page to return transparent pixel..."
uci set uhttpd.main.error_page="/1.gif" && uci commit
elif [ -s "/usr/sbin/httpd_gargoyle" ]
then
# Write without testing
echo "httpd_gargoyle found..."
echo "updating server error page to return transparent pixel..."
uci set httpd_gargoyle.server.page_not_found_file="1.gif" && uci commit
else
echo "Cannot find supported web server..."
fi
fi
}
update_blocklist()
{
#Delete the old block.hosts to make room for the updates
rm -f /etc/block.hosts
# Correct endpoint for transparent pixel response
if [ "$TRANS" = "Y" ] && [ -e "/www/1.gif" ] && ([ -s "/usr/sbin/uhttpd" ] || [ -s "/usr/sbin/httpd_gargoyle" ])
then
ENDPOINT_IP4=$(uci get network.lan.ipaddr)
if [ "$IPV6" = "Y" ]
then
ENDPOINT_IP6=$(uci get network.lan6.ipaddr)
fi
fi
echo 'Downloading hosts lists...'
#Download and process the files needed to make the lists (enable/add more, if you want)
wget -qO- http://www.mvps.org/winhelp2002/hosts.txt| awk -v r="$ENDPOINT_IP4" '{sub(/^0.0.0.0/, r)} $0 ~ "^"r' > /tmp/block.build.list
wget -qO- "http://adaway.org/hosts.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- http://www.malwaredomainlist.com/hostslist/hosts.txt|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- "http://hosts-file.net/.\ad_servers.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#Add black list, if non-empty
if [ -s "/etc/black.list" ]
then
echo 'Adding blacklist...'
awk -v r="$ENDPOINT_IP4" '/^[^#]/ { print r,$1 }' /etc/black.list >> /tmp/block.build.list
fi
echo 'Sorting lists...'
#Sort the download/black lists
awk '{sub(/\r$/,"");print $1,$2}' /tmp/block.build.list|sort -u > /tmp/block.build.before
#Filter (if applicable)
if [ -s "/etc/white.list" ]
then
#Filter the blacklist, supressing whitelist matches
# This is relatively slow =-(
echo 'Filtering white list...'
egrep -v "^[[:space:]]*$" /etc/white.list | awk '/^[^#]/ {sub(/\r$/,"");print $1}' | grep -vf - /tmp/block.build.before > /etc/block.hosts
else
cat /tmp/block.build.before > /etc/block.hosts
fi
if [ "$IPV6" = "Y" ]
then
safe_pattern=$(printf '%s\n' "$ENDPOINT_IP4" | sed 's/[[\.*^$(){}?+|/]/\\&/g')
safe_addition=$(printf '%s\n' "$ENDPOINT_IP6" | sed 's/[\&/]/\\&/g')
echo 'Adding ipv6 support...'
sed -i -re "s/^(${safe_pattern}) (.*)$/\1 \2\n${safe_addition} \2/g" /etc/block.hosts
fi
}
restart_firewall()
{
echo 'Restarting firewall...'
if [ -s "/usr/lib/gargoyle/restart_firewall.sh" ]
then
/usr/lib/gargoyle/restart_firewall.sh > /dev/null 2>&1
else
/etc/init.d/firewall restart > /dev/null 2>&1
fi
}
restart_dnsmasq()
{
if [ "$1" -eq "0" ]
then
echo 'Re-reading blocklist'
killall -HUP dnsmasq
else
echo 'Restarting dnsmasq...'
/etc/init.d/dnsmasq restart
fi
}
restart_http()
{
if [ -s "/usr/sbin/uhttpd" ]
then
echo 'Restarting uhttpd...'
/etc/init.d/uhttpd restart
elif [ -s "/usr/sbin/httpd_gargoyle" ]
then
echo 'Restarting httpd_gargoyle...'
/etc/init.d/httpd_gargoyle restart
fi
}
restart_cron()
{
echo 'Restarting cron...'
/etc/init.d/cron restart > /dev/null 2>&1
}
remove_config()
{
echo 'Reverting config...'
# Remove addnhosts
uci del_list dhcp.@dnsmasq[0].addnhosts=/etc/block.hosts > /dev/null 2>&1 && uci commit
# Remove cron entry
sed -i '/adblock/d' /etc/crontabs/root
# Remove firewall rules
sed -i '/--to-ports 53/d' /etc/firewall.user
# Remove Tor workarounds
uci del_list dhcp.@dnsmasq[0].server > /dev/null 2>&1 && uci commit
uci set dhcp.@dnsmasq[0].noresolv='0' > /dev/null 2>&1 && uci commit
# Remove proxying
uci delete uhttpd.main.error_page > /dev/null 2>&1 && uci commit
uci set httpd_gargoyle.server.page_not_found_file="login.sh" > /dev/null 2>&1 && uci commit
}
toggle()
{
# Check for cron as test for on/off
if grep -q "adblock" /etc/crontabs/root
then
# Turn off
echo 'Turning off!'
remove_config
else
# Turn on
echo 'Turning on!'
add_config
fi
# Restart services
restart_firewall
restart_dnsmasq 1
restart_http
restart_cron
}
#### END FUNCTIONS ####
### Options parsing ####
case "$1" in
# Toggle on/off
"-t")
toggle
;;
#First time run
"-f")
install_dependencies
add_config
update_blocklist
restart_firewall
restart_dnsmasq 1
restart_http
restart_cron
cleanup
;;
#Reinstall
"-r")
remove_config
install_dependencies
add_config
update_blocklist
restart_firewall
restart_dnsmasq 1
restart_http
restart_cron
cleanup
;;
#Default updates blocklist only
*)
update_blocklist
restart_dnsmasq 0
cleanup
;;
esac
#### END OPTIONS ####
exit 0
#/etc/black.list
#add some server that the list doesn't block
example1.block.com
#/etc/sysupgrade.conf
#This file is a list of files that should be preserved through upgrades
#OPTIONAL!!!!!
/etc/passwd
/etc/shadow
...
...
/etc/adblock.sh #ADD THIS LINE
/etc/white.list #AND THIS ONE
/etc/block.hosts #AND THIS ONE
/etc/black.list #AND THIS ONE
#/etc/white.list
#Add whitelisted addresses, when appropriate, etc.
a248.e.akamai.net

stoinov commented Feb 23, 2014

Instead of dnsmasq restart you could just use killall -HUP dnsmasq which will reload the host files as per http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html#lbAF

Cybso commented May 21, 2014

Have a look at my fork at https://gist.github.com/Cybso/bf9b69c6a638ffd68281

  • Use 0.0.0.0 instead of 127.0.0.1 to avoid conflicts with local webservers
  • Added IPv6 support (::, which means "not specified")
  • adaway.org redirects to https, now.
Owner

teffalump commented May 24, 2014

yeah, I just noticed that adaway thing gotta update it, lol. yeah, 0.0.0.0 would work; i'll update that stuff thanks! =-)

Owner

teffalump commented May 24, 2014

Issue about SIGHUP is that it re-reads addnhosts, hosts, etc files, but it doesn't re-read the config. If this is the first time running the script, dnsmasq would not update and read the new addnhost file. I probably should extend the script, so it can use pkill, not simply restarting, when applicable because I agree, re-reading the files is much better than restarting the process.

Owner

teffalump commented May 24, 2014

Modified script to use pkill when possible.

Owner

teffalump commented May 24, 2014

Added firewall restart if settings changed, since that's required on first run.

borore commented Jun 17, 2014

Great! thank you!

Owner

teffalump commented Jul 8, 2014

Added checks for pkill command and iptables-mod-nat-extra package.

ronison commented Jul 16, 2014

Since Barrier Breaker / Trunk version the pkill command is in procps-pkill package and no more in procps package.

echo 'Installing procps package...'
opkg install procps > /dev/null
opkg install procps-pkill > /dev/null #ADDING THIS LINE WILL SOLVE

Owner

teffalump commented Jul 17, 2014

Thanks a bunch. Updated. I'm running gargoyle, so I'm somewhat out of step with the main openwrt releases.

troych commented Sep 8, 2014

The ipv6 support does not seem to work for me, shouldn't it create the "/tmp/block.build.before" file including "::" and "0.0.0.0" after doing the sed command?

edit: googled a bit and it looks like this could be the problem: http://osdir.com/ml/bug-gnu-utils-gnu/2011-04/msg00002.html and indeed if I change line 77 to this:

sed -i -re 's/^(0\.0\.0\.0) (.*)$/\1 \2\n:: \2/g' /tmp/block.build.before

I get my ipv6 supported blocklist

Owner

teffalump commented Sep 30, 2014

@troych: Thanks for this. I'll update it. Was working for me, but, hey, more readable that way, too. Tons of switches squashed together... eww!

Owner

teffalump commented Sep 30, 2014

Updated.

great job.
works fine on my router.
is possible to separate the ad-blocking in the lan and wifi?
for example no ad-block on the LAN and enable ad-blocking in the WIFI?
thank you

Owner

teffalump commented Nov 26, 2014

@norman25: Sorry about the delay responding! I think you are saying that you want no ad-blocking on the wired internet, but blocking on wireless?

Owner

teffalump commented Nov 26, 2014

Updated. Added wireless only support.

norman25 commented Dec 2, 2014

@teffalump: you want no ad-blocking on the wired internet, but blocking on wireless?
yes is exactly that i need!!!
@teffalump: Updated. Added wireless only support.
well i'm testing now thx.

norman25 commented Dec 2, 2014

well I tried the script and it works but recently in firefox, opera and chrome blocking are skipped. (with internet explorer and slimbrowser working very well).
any idea why this happens?

Owner

teffalump commented Dec 8, 2014

@norman25: Sorry again about delay, busy IRL. Do you mean the wireless-only portion is broken? Or blocking in general is broken?

Owner

teffalump commented Dec 8, 2014

Updated. Disabled some hosts urls. Moved ipv6 support after whitelist function.

@teffalump: Sorry again about delay, busy IRL. Do you mean the wireless-only portion is broken? Or blocking in general is broken?
nope your modified wireless version is working fine. there is a problem in some pages that use https. example, (https://myadvert.ads) seems not blocked correctly, anyway only occurs in some pages is not a big deal.
thx a lot.

Tremendously well done : thank you very much !!!
Would there be an easy way to not have all blocked url go to an empty gif like in http://pakitong.blogspot.ch/2014/02/openwrt-installing-adblock-with.html (but without creating another interface) ?

Owner

teffalump commented Dec 22, 2014

@norman25: Odd. Maybe explicitly add them to the blacklist? Hehe, or ignore them.

Owner

teffalump commented Dec 22, 2014

@gregoirefavre: I think you are asking, is there a simple way to return an empty file instead of returning a spoofed DNS response (like, 0.0.0.0)? Put simply, it's complicated (as you can tell from the article). In this case, the adblocking is done on the router by returning an empty page, which complicates the setup. However, one could run a pixelserv setup on the personal computer and do the same thing. The bigger issue seems to be how to accomplish the best adblocking setup. Adblock Edge, ABP, etc. work very well in the browser, although there are downsides. Then there is this way for the router. Then there is running dnsmasq + pixelserv on the computer. Etc. A complete solution probably incorporates some of everything. I'll look into simpler setups to return an empty page, but I think any solution will be somewhat inelegant. With the spoofed DNS response, I haven't run into performance issues - curl 0.0.0.0:80 or 0.0.0.0:443 immediately refuse connections, they do not timeout. To me, this setup seems good enough.... What is it? The best is the enemy of the good.

flashjh commented Dec 25, 2014

This is really awesome! I used opkg install wget and then everything worked perfectly. I have this combined with OpenDNS on my Dlink DIR-825 with OpenWRT. Finally, a decent solution for some control of the home network. Thanks!!

Hi, really appreciate your work on this. I have managed to implement it such that the script works, however i am having trouble with the white.list implementation.
This is ok because i CURRENTLY have no use for this, but in the future it may be a problem.

I find that when i have a file called white.list in /etc then the script still executes without error, however block.hosts ends up completely empty.
If white.list is not present, block.hosts populates as expected.

I have no experience with shell scripting so i unfortunately can't add any insight.
Regards

Owner

teffalump commented Jan 2, 2015

@lantis1008: Yeah, I think what happens is that grep matches white characters (like space) in the blocked urls, so every line is white-listed. I've added a regex to hopefully remove all whitespace only lines. I'll try to test it. =-/

@flashjh: Glad you like it! Keep me up-to-date on the openwrt stuff, I run gargoyle, so I'm a little out-of-step, so if anything breaks, tell me. =-)

Owner

teffalump commented Jan 4, 2015

Added GNU wget check (not Busybox) and install.

Hi, I'm using OpenWrt r36088 and when I run " sh /etc/adblock.sh " i get:
" /etc/adblock.sh: line 132: syntax error: unexpected end of file (expecting "then") "
Any tips?
Thanks

Owner

teffalump commented Jan 5, 2015

@rodtrevizan: Hehe, I've had that before, too. I believe that the error occurs when the file is saved in dos format, not unix. Try running something like this:

root@192.168.1.1: /etc# sed -i -re 's/\r$//g' adblock.sh

The command removes the dos end-of-line character (\r). If that doesn't work, fiddle around with trying to convert the file to unix format. Hope that helps.

Owner

teffalump commented Jan 5, 2015

Removed pkill, just use killall - fewer package issues.

Thanks for updating it to remove those package issues. just updated my gargoyle to 1.7.0 and it spat issues at me until this change :)

Owner

teffalump commented Jan 5, 2015

@lantis1008: Yeah, same thing, lol. =-)

Owner

teffalump commented Jan 5, 2015

Tried to better handle wget installation and checking. IPV6 support optional now.

Thanks for all, and it's true they aren't any perfomance issue I can when using it (which is all the time) :-)

pkrexer commented Jan 7, 2015

Hello, I'm running on Chaos Calmer r43823 and I'm running to this error when executing the script:

* satisfy_dependencies_for: Cannot satisfy the following dependencies for iptables-mod-nat-extra:
*      kernel (= 3.14.27-1-7c18d64c08713497f02c692f0b8572ab) *
* opkg_install_cmd: Cannot install package iptables-mod-nat-extra.

However, it still seems to be working? I've tested a number of websites and the ads are indeed being blocked.

Owner

teffalump commented Jan 8, 2015

@pkrexer: You tried looking at development package snapshots? Also, /etc/opkg.conf should show where the packages are coming from... and since you are running a very new version, may not match with the package release. shoulder shrug No idea, other than that. Hehe, if it's working... is kind of annoying though. It sounds like it's saying it isn't installed, but is... =-/

Owner

teffalump commented Jan 27, 2015

Use restart_firewall.sh for those Gargoyle users - nice script that won't mess up other things. Thanks @lantis1008.

great solution! :)

triplefb commented Feb 6, 2015

This is awesome, thanks so much. Has anyone had any issues with Chrome on IOS while running this script? It seems to just hang, both on my phone & ipad. (I'd much rather use Safari with this than Chrome without though!)

@teffalump: i'm looking at implementing this into a gargoyle plugin with your permission.

Owner

teffalump commented Feb 16, 2015

@lantis1008: Yeah, go right ahead. It was on my to-do list, but... yeah. Lol. =-)

Owner

teffalump commented Feb 16, 2015

@triplefb: I've had reports about some issues. I guess I don't know, really. I wonder if the browser must contact certain servers...? I really don't know.

norman25 commented Mar 2, 2015

@teffalump
i see you change the script, any improvements?

tinyema commented Mar 18, 2015

@teffalump
great job, it works perfectly!
i found a couple of mistakes...
Filtering white list:
egrep -v "^[[:space:]]$" /etc/white.list | awk '/^[^#]/ {sub(/\r$/,"");print $1}' | grep -vf - /tmp/block.build.list > /etc/block.hosts
must be:
egrep -v "^[[:space:]]
$" /etc/white.list | awk '/^[^#]/ {sub(/\r$/,"");print $1}' | grep -vf - /tmp/block.build.before > /etc/block.hosts
and you forgot to delete /tmp/block.build.before in cleaning up's section:
rm -f /tmp/block.build.before

First, thanks for the script! With my router (TL-WR184N), I had trouble when it got to the installing wget (with ssl) part (not enough memory). I changed the https to http and it retrieved the list without error. I do have a question though... instead of ads, I get the "Server not found". Was that how it is suppose to be or is it suppose to return a blank placeholder?

to add to @tinyema's suggestion, the else statement following that line should also have "/tmp/block.build.before" if i'm not mistaken.

@prouderthings yes it is supposed to show "server not found". There are ways of returning a blank placeholder,

@teffalump
Is it possible to use a variable instead of "0.0.0.0" in case i wish to specify a destination other than this for redirecting? i suck with awk and all my attempts so far don't work. my main issue is the pattern matching at the end "/^x.x.x.x/".

EDIT -> got it. check out my fork if you like

That works well for me @lantis1008, thanks! It really seems to pound the cpu on my router though. (wndr3700v2, gargoyle 1.7.0.) Sometimes pages display the login screen to my router instead of the ad element too. (Which is still better than the ad, ha ha.) Thanks again all!

@triplefb if its displaying the login screen, did you correctly make the substitution in /etc/config/httpd_gargoyle option error_page_not_found_file to "/1.gif". you'll then need to wget the 1.gif image into your /www/ directory. check out my fork for more details :)

please be aware that in the next gargoyle version (1.7.2) it is likely that this substitution will have to be changed as it is being migrated from httpd to uhttpd. i'll keep you posted.

Whoops I didn't read closely enough, missed the part about updating /etc/config/httpd_gargoyle. Thank you!

There is something you have to keep in mind when you use this script:
It redirects DNS requests and filters out the balck listed hosts.

However, if you need to update bind zones, e.g. because you are running a ddns setup from or behind the router using the adblock script, you may screw up your dyndns updates.

In order to let dns updates through, something like

BIND_REQUESTS="Y"
BIND_REQUEST_HOST=192.168.0.2 # host issuing ddns updates
...
and
if [ "$BIND_REQUESTS" == "Y" ]
then
FW1="iptables -t nat -I PREROUTING ! -s $BIND_REQUEST_HOST -p tcp --dport 53 -j REDIRECT --to-ports 53"
FW2="iptables -t nat -I PREROUTING ! -s $BIND_REQUEST_HOST -p udp --dport 53 -j REDIRECT --to-ports 53"

or something like this.

@lantis1008: Does your fork include @tinyema's changes?

@keiler3000 yes my fork includes the changes from @tinyema
I also changed the firewall restart script section so that it only applies to gargoyle. I basically removed the if statement.

Also I've had no issues with my ddns updating, but if you can sell me a good reasoning behind it I'll try adding it in.

Well, the reasoning is:

I have a setup like this:

client -//WiFi/LAN//- router -//Internet//- self hosted domain host

After activating adblock.sh on the router I was unable to update zone entries using nsupdate from the client on the host. I always got TSIG errors and in the end the update failed with BADKEY. From other hosts, e.g. outside my LAN, there was no problem. After hours of tying I realized that the script might be the cause as it, obviously, acts on port 53. After including the exception for the client IP, the problem was gone.

9M2PJU commented Mar 27, 2015

Thanks

timche commented Mar 31, 2015

Thanks for the script. I've installed it on my TL-WR1034ND with ONLY_WIRELESS="Y". It works fine, but my LAN devices are affected as well and I don't know why.

Owner

teffalump commented Mar 31, 2015

Hey guys, sorry I've been out of it. IRL stuff. Let me get those edits started. Lol. And thanks @lantis1008 again. =-)

Owner

teffalump commented Mar 31, 2015

@norman25: Thought I responded back then. But anyways, nothing really major. I'll take the suggestion and add the config for the error page, I think. =-)

Owner

teffalump commented Mar 31, 2015

@tinyema: Thanks, another mistake (cat /etc/block.build.list > /etc/block.hosts)... updated that, too. Thanks!

Owner

teffalump commented Mar 31, 2015

@lantis1008: I actually was thinking the same thing about the variable holding the redirection. I like what you did with that (and the pixel). Cool if I add it? =-)

Owner

teffalump commented Mar 31, 2015

@tch3ung: Hmm... Hmm... I don't know. =-/ Maybe the interface isn't named that? Have you looked?

Owner

teffalump commented Mar 31, 2015

Hopefully the ip6 endpoint choice works. Gotta test it. Regexes.... uggh.

No worries man add away. :)

My script does actually work I've found, I was just being foolish in my testing.
You can make it universal by doing a similar thing to your firewall IF.
Test for the file httpd_gargoyle. If yes, do a uci set httpd_gargoyle.something.error_page "/1.gif"
Otherwise do a uci set uhttpd.uhttpd.error_page "/1.gif"

I'm currently at work so I don't have access to my router to get the actual file and command names sorry mate.

If you don't want to boost the script you can make people do it themselves :)

Owner

teffalump commented Apr 1, 2015

@lantis1008: Thanks! Hehe, the hands-off approach is tempting. Lol. I suppose I'll do it anyways though. =-)

Owner

teffalump commented Apr 1, 2015

A first attempt at supporting both servers. Gotta test.

Missing an extra 'fi' between line 107 and 108. Crashes otherwise.

Code looks good apart from missing fi as @OpenWrtFan said.
Only issue I can think of is if a person does a sysupgrade with attempt to preserve files then they will end up with both files. May need a nicer way of testing, or could edit both.

If you want to test for those settings first you could do an AND test using uci get.

Owner

teffalump commented Apr 1, 2015

@OpenWrtFan: Thanks!

@lantis1008: You mean, the uhttpd or httpd_gargoyle thing? I could switch the order, so uhttpd is always first, then only ever updates uhttpd if it has it? Or, is it something else?

Testing for uhttpd first would fix the issue :)

blocking lists used with abp or ublock (e.g. easylist) have some nice entries to block youtube ads, social tracking and filters for non-english ad networks. any chance this could be included here?

thanks for putting this together.

Easy list is designed in a format that works specifically with those ad blockers and would take a lot (too much) filtering to get into here IMO.

Owner

teffalump commented Apr 2, 2015

Moved order of server checks... hopefully should obviate any upgrade problems.

Owner

teffalump commented Apr 2, 2015

@Schabernack: Yeah, I thought about how to work with the lists, but I completely agree with @lantis1008. This method is leaky, but takes care of most unwanted domains. Those blocklists mostly work on 'objects' (e.g., /*.block.js) within the page. That kind of fine blocking should not happen on the router. Over TLS, it's not even possible (I think?!). This method of DNS blacklisting will, hopefully, become ineffective once dnscrypt/dnssec becomes more widely used. For non-browser addons, Polipo, or Privoxy, can do things like the fine blocking you are talking about. However, thanks for the interest! =-)

I still believe it would be good to include an option to explicitly exclude a specific host or IP range from the filtering in the firewall rules.

Owner

teffalump commented Apr 5, 2015

@keiler3000: Yeah, I have no problems with adding another check thingy for that.

Owner

teffalump commented Apr 5, 2015

@keiler3000: Added an ip range exemption. I don't know if it will work, lol.

@ghost

ghost commented Apr 12, 2015

Hello,

@teffalump: privoxy can't run as intercepting/transparent proxy for port 443 (HTTPS) though, so you can't block https sites, and you can't filter YouTube ads with it since YouTube is https (though dnsmasq can't either, of course). Anyway, I need a solution that works out of the box for every device connected, since I want to keep my net clean. dnsmasq is way simpler in this regard, and easy to script (I wrote my own implementation, resulting in more than 70000 entries -- works without any problems on a WDR4300). So we're left with null routing those malware and ad domains if dnssec/dnscrypt becomes a reality; iptables is too slow for several thousand sites, I'm afraid. Any other alternative?

Great work on your script, though, it was nice to see a different implementation.

Owner

teffalump commented Apr 15, 2015

@1ems: Hiya! Yeah, what I meant was that over tls, the interception/filtering could not happen on the router. For example, I run dnscrypt + dnsmasq + polipo + adblock (in the browser), so I can still filter https sites but it must happen after the tls is stripped. I'm not completely understanding what you are saying about alternatives, the basic config for the script uses dnsmasq to null route. Or you mean that iptables intercepts the requests? Or? I hope that dnssec/crypt do become a reality! =-) Thanks!

@teffalump in order to iptables range to work you need to check for "xt_iprange" module, and if it's not loaded, install iptables-mod-iprange and modprobe xt_iprange, or iptables will simply ignore the extemption rules.

Owner

teffalump commented Apr 21, 2015

@aleblack: Hahah, thanks! Man, checking for these packages....uggh. =-) Unless it's a non-standard install location, I don't believe the modprobe is necessary, no?

hello dear,
i'm noob running gargoyle 1.7.1
i've tried to add all the way you suggested
now im confuse how i can know this script working or not
any way is there a log/report error when it's not run correct?
thanks alot

*edit
root@Gargoyle:~# sh /etc/adblock.sh
iptables-mod-nat-extra is installed!
Updating package list...
Installing wget (with ssl)...
ERROR: Not enough space in destination root to install specified packages:
libpthread, zlib, libopenssl, wget, librt, libpcre

Updating config, if necessary...
Downloading hosts lists...
Adding blacklist...
Sorting lists...
Filtering white list...
Cleaning up...
Restarting dnsmasq...

which one do i need fix?
im using wr740

Hey teffalump. What's got you not recommending the pixel serving idea?
Just out of curiosity. Maybe you've thought of something I haven't :)

Owner

teffalump commented Apr 24, 2015

@lantis1008: Hahah, no. I just think many people won't understand what's happening behind the scenes, and it could lead to non-obvious errors. I'll remove the recommendation. =-)

Owner

teffalump commented Apr 24, 2015

@brainstromer: Well, sounds like there is no space left to install the packages for wget w/ ssl. Odd. What's taking so much space? Use df and du. You don't actually need wget with ssl if you use non-ssl urls. Yeah, better error handling is on my to-do list.

how do i turn it off again?

norman25 commented Jun 2, 2015

@teffalump: why you use (https://adaway.org/hosts.txt).?
I'm use (http://adaway.org/hosts.txt) and works exactly in the same way!
I'm think is no needed to install wget ssl package and waste the very limited space in the router!

Owner

teffalump commented Jun 6, 2015

@norman25: Yeah, I thought that it redirects to https. But I think you're right. Weird. Maybe they changed it? Lol. I'll update it to be optional for ssl. Thanks.

Owner

teffalump commented Jun 6, 2015

@manaueruel: A few changes. 1) Remove the cron entries, 2) Remove the firewall additions, 3) Remove the dhcp change, and, if necessary, 4) Undo the pixel serving. Then restart the firewall and dnsmasq (and possibly the httpd server). You think I should add an option to revert the changes? Like, ./adblock.sh -r would undo all the changes?

@manaueruel: how do i turn it off again?
@teffalump: would undo all the changes?
I think he wants something to lock on and off adblock in the router like ad-toggle.sh (on - off)

lacek commented Jun 13, 2015

If TRANS is set to "Y", there's error about missing ]:

...
Updating config, if necessary...
sh: missing ]
httpd_gargoyle found...
...

^ looks like its line 125.

Owner

teffalump commented Jun 21, 2015

@lacek, @lantis1008: Thanks!

Owner

teffalump commented Jun 21, 2015

@norman25: Hmm... The toggle thing should be possible, but I'd have to add options (I think) because the cron entry must only update the block list. For example, ./adblock.sh = toggle on/off, ./adblock.sh -u = update block list (update dnsmasq), ./adblock.sh -f = first time use (adds config, too). Or something like that. Thoughts?

Owner

teffalump commented Jun 21, 2015

Just added some command line options. -f is for running it the first time, -t toggles it on/off, and default just updates the blocklist. Separated code into functions. Gotta test it, though. Lol.

@teffalump:
i think this is not more Simple OpenWRT adblock implementation is more a Advanced OpenWRT adblock implementation right now!! jaja!!
fanstastic work man!!

Hi!
Sorry my bad english!

When i run first your script (on my gargoyle router), the ip in the host file is: 192.168.1.1. It's OK (TRANS=Y in my config).
When i update the blocklist manually, the ip in the hostfile is: 0.0.0.0
Why? is this ok?

Owner

teffalump commented Jun 29, 2015

@hunsaman: If TRANS=Y, then IP should be 192.168.1.1, and the update should keep the ip as 192.168.1.1. I think it is a bug. Let me look into it. Thanks.

UPDATE: I fixed the issue (I'll do more testing, too). Basically, the update_blocklist was not using the correct endpoint_ip6 and endpoint_ip4 values.

Easier way:

# Redirect endpoint
if [ "$TRANS" == "Y" ] && [ -e "/www/1.gif" ] && ([ -s "/usr/sbin/uhttpd" ] || [ -s "/usr/sbin/httpd_gargoyle" ]); then
    ENDPOINT_IP4=$(uci get network.lan.ipaddr)
    if [ "$IPV6" == "Y" ]; then
        ENDPOINT_IP6=$(uci get network.lan6.ipaddr)
    fi
else
    ENDPOINT_IP4="0.0.0.0"
    ENDPOINT_IP6="::"
fi
Owner

teffalump commented Jun 30, 2015

@norman25: Haha, that's probably true. Its getting a little complex, lol. But, the simplest use is pretty easy to use, so... hehe. Maybe I'll update description. =-)

Owner

teffalump commented Jul 3, 2015

@hunsaman: Yeah, could do it that way, more checks, etc. Although the 'else' clause is overwriting the config value which is a no-no =-)

UPDATE: I added your checks without the else clause. =-)

sleepy1 commented Jul 5, 2015

Hey, nice script. Just one question, how does the only use wifi blocking work. It seems to block ads on all interfaces even with this option turned on.

Owner

teffalump commented Jul 5, 2015

@sleepy1: Hehe, it's suppose to work! Ideally, the firewall rule should only redirect DNS requests on wlan+ interface(s), that is, wireless interfaces. However, I have heard of issues. I don't really know why it doesn't work sometimes, maybe there is bridging, misleading names, etc. Any suggestions are welcome. Also, look in /etc/firewall.user for multiple conflicting rules - there should be only two DNS rules, one for udp, one for tcp. Thanks! =-)

Owner

teffalump commented Jul 5, 2015

Updated to use '=' instead of '==' for portability reasons. Although, I'm unsure if it's necessary.

I finished my plugin based on your work mate, sorry i forgot to tell you!
You can check it out here:
http://www.gargoyle-router.com/phpbb/viewtopic.php?f=14&t=7249

i have of course credited you in the post and the source code. Many thanks.

Owner

teffalump commented Jul 6, 2015

@lantis1008: AWESOME! SO AWESOME!!! =-) Great work! Sorry if I'm missing something, but is there a place for development? Like, push onto gh, bb, etc? =-)

I've got a private git setup for it. In the next week or so I'll probably push it to master to make it public. I keep a pretty close eye on this thread anyway so I haven't missed anything yet :)

Owner

teffalump commented Jul 9, 2015

@lantis1008: Hehe, great to hear! I was more asking for some of the other people who have made suggestions and comments (seems like many run gargoyle) and may want to help with the plugin. =-)

Owner

teffalump commented Jul 9, 2015

Added option "-r" to fully reinstall since some config changes require it.

Hi teffalump, love your script, works very well and is quite elegant solution. What I miss is the possibility to import my adblock custom rules. Do you think it might be possible to integrate this syntax to your script so we can add adblock rules to the "black.list"? => https://adblockplus.org/en/filters

roger- commented Jul 17, 2015

@teffalump

Have you considered submitting this as an OpenWrt package?

Owner

teffalump commented Jul 20, 2015

@miniminime: Put simply, no. =-/ Those rules match against page objects, like '/ad-plugin.js'. This setup can't do that, it just isn't working at that level. You'd have to run something like privoxy on the router, MITMing SSL connections, etc. I think that more granular blocking should be done on the computer (e.g., privoxy) or in the browser (e.g., ublock/adb/etc).

@roger-: I had briefly looked at it, but was somewhat scared about my lack of knowledge. If you are game, feel free to do it. =-) I'll look at it more because there does seem to be a good amount of interest. And now with the gargoyle plugin... =-)

roger- commented Jul 20, 2015

@teffalump

I'll have a look and see how that might work. Until then I'd suggest moving this to GitHub to make collaboration easier.

Hi Teffalump,

Awesome piece of software you created. Many thanks.

However, for some reason it is not working for me. I've installed the scripts and ran the first install part.
I ran into storage issues on the main flash, so I changed the scripts to work with my USB disk instead.

The firewall.user is changed
The block.list is added to dhcp config
The dnsmasq service was restarted
The firewall was restarted
I removed my dns server settings and put them in default state

The client pc has my router's IP as DNS server address.

When I open analytics.google.com (in Chrome), it just opens the site even though the block.list says that it should be routed to 0.0.0.0

It seems that dnsmasq is not processing the adblock stuff at all.

Any ideas what I can check?

logread tells me this:
daemon.err dnsmasq[32666]: failed to load names from /mnt/usbdisk/adblock/block.hosts: Permission denied

Any thoughts on how to get around this?

logread tells me this:
daemon.err dnsmasq[32666]: failed to load names from /mnt/usbdisk/adblock/block.hosts: Permission denied

Problem solved after moving the block.hosts file to the /etc folder

daemon.info dnsmasq[472]: read /etc/block.hosts - 56813 addresses

Now I need to figure out how to solve the permission issue with the USB drive.

Figured out that the file can be read by dnsmasq if the entire chain of folder names can be read by "nobody". Set file permissions to xx5.

Owner

teffalump commented Jul 23, 2015

@hansvdmarel: Sorry I didn't get back to you quickly. Also glad to hear you solved it. Folder/file permissions can be trick sometimes. =-)

Owner

teffalump commented Jul 23, 2015

Another thing to add to the script is restarting cron. Otherwise it won't actually update via cron.

/etc/init.d/cron restart

^ that's not strictly true, and certainly not in this case in my experience.

Well, for OPENwrt it is really :)

Anyway, found another issue.

While updating the lists via cron, sometimes the router crashes. I think this has something to do with the device running out of memory.
I've also noticed a crash when running the script manually, so I'm quite confident that this is the issue.

I believe the issue manifests itself when the sorting is executed. How important is the sorting function for the operation? Can we do without it?

Owner

teffalump commented Jul 30, 2015

@hansvdmarel, @lantis1008: I can add the cron restart. I've had no issues, but nothing too onerous. The sorting is a unique sort... and since there is significant overlap between lists, I'm hesitant to undo that. Thoughts?

Currently the sorting function also does the unique as well as far as i can figure out.
You can do a non sorting style of unique, but i don't think it is any less resource intensive.

We are talking about ~ 700kB in RAM (with ipv4 only) currently (unsorted list + sorted unique list). ipv6 is probably double this approximately (bit less because of only unique values).
~1.4MB of RAM... for me personally i have plenty of space. But smaller devices might run into this issue.

1 solution might be to remove "/tmp/block.build.list" as soon as you have done the sort, effectively halving the space we are using.
i.e. ~700kB in RAM, down to ~350kB after removing "/tmp/block.build.list", then potentially doubled due to ipv6 back to ~700kB.
It's not perfect, but it is a start. Achieved by either passing a value to the cleanup function, or just straight up rm -f'ing the file, and then the cleanup function only has to remove 1 file.

@teffalump I'm currently talking to a guy who is having a conflicting issue with your script + Tor.
They create conflicting iptables rules and .onion sites no longer get resolved.

The Tor rules are:
iptables -t nat -A tor_client -p tcp --dport 53 -m string --hex-string '|xxxxxxxxxxxx|' --algo bm -j REDIRECT --to-ports 9053
iptables -t nat -A tor_client -p udp --dport 53 -m string --hex-string '|xxxxxxxxxxxx|' --algo bm -j REDIRECT --to-ports 9053

Is there any way of making these rules play nicely or are they completely incompatible?

Owner

teffalump commented Aug 11, 2015

@lantis1008: Sorry about inactivity. IRL stuff. The sorting thing is... I can remove the old list. On the iptables rule, hmm... that's an interesting question. Those rules are doing string matches against the content, then routing them, I wonder if it is possible to similarly do a narrower DNS match for the rules. Thoughts?

PS: I'm trying to think this through, basically, we want to pass through .onion sites. Could we do an inverse string match against '.onion' sites? Like, just subsume the tor rules into the default rule, but the inverse.

PPS: E.g., -m string ! --hex-string '|xxxxxxxx|' --algo bm

Owner

teffalump commented Aug 11, 2015

@lantis1008: I added your workaround for Tor, but I'm still thinking it should be possible to work around it in the rules themselves.

Yeah I'd love to see a proper workaround for it but I can't figure it out. Currently what I implemented works. But it's ugly.

Owner

teffalump commented Aug 14, 2015

@lantis1008: Why don't we re-route the non-found domains to the tor port? That is, clobber the Tor rule, but then have dnsmasq forward the "dns-miss" requests through the tor client.

Owner

teffalump commented Aug 14, 2015

Yeah, I think what I think can be done is set dnsmasq to noresolv and server to tor DNS port. This should sit dnsmasq in front of the tor DNS resolver, but keep the caching abilities.

Owner

teffalump commented Aug 14, 2015

First attempt at working around Tor.

I've sent you a PM on the gargoyle-router website regards an issue.

Owner

teffalump commented Sep 10, 2015

@lantis1008: Yeah, I'm going to open an issue on the github page. Thanks! =-)

dcavni commented Sep 29, 2015

First off all i must say that this is excellent script. It hunts ads very well :)

I have a question regarding how this works with OpenVPN. I set up a server running on
10.8.0.0 255.255.255.0. (http://wiki.openwrt.org/doc/howto/vpn.openvpn)
It works ok, until i run the script. Then traffic destinated to internet stops working on client. Local traffic still works.

When i use the option to exclude 10.8.0.1 - 10.8.0.254 from adblock.sh then traffic destinated to internet starts to work again (with ads).

Any idea what could be wrong?

Owner

teffalump commented Oct 1, 2015

@dcavni: Thanks! Let me see if I understand. You setup a VPN, it works great. Then the script breaks it. =-( Are you connecting from the LAN side? Or WAN? Furthermore, can you ping past the router? That is, does...

# ping 8.8.8.8

work... or does it timeout? If it works, then can you see if you can resolve...

# dig google.com

I'm wondering if there is some firewall rule conflict going on.

dcavni commented Oct 3, 2015

Well it doesn't completly brake the tunnel, local traffic still works. I did some testing now and found out, that if i make a OpenVPN connection on a computer then everything works. If i use the same config file on a phone (Android), then only local traffic works.

If i exclude VPN ip range from script, then Internet and local traffic works on a phone and also on computer as before (with ads)

Now i'm not completly shure what is wrong. Ping works but very bad. Only 3 packets out of 4 returned, I cannot try dig command on Windows.

Any sugestions?

Edit: Ok, today i did some more testing. If i change Android OpenVPN app for OpenVPN client app (Two different apps). Then web browsing works on android. It look's like original OpenVPN app doesn't work well when adblock.sh works on router. Also i see ads on phone so it look's like the script isn't blocking ads on a 10.8.0.0 / 255.255.255.0 (default OpenVPN tun).

@dcavni and @teffalump
The Problem with the non working Internet connection for OpenVPN Clients is because of a dnsmasq config. By default DNS queries from a non local subnet are not allowed. This is be done by "option localservice '1'" in /etc/config/dhcp
If you switch this value to "0" and run 2 times "sh /etc/adblock.sh -t" all VPN clients should reach the internet. You also can extemp the VPN Range in the adblock.sh script, but then adblock wont work for VPN. I described this also in my Blog Post "http://blog.doenselmann.com/werbung-direkt-auf-openwrt-router-blocken/" (but its in german).
I hope this will help you.

dcavni commented Oct 4, 2015

It works :) Just perfect. Thanks to everyone for help.

@MichiMunich
Is that the same reason for Tor not working do you think?

Owner

teffalump commented Oct 6, 2015

I'm trying to get a handle on what is happening...

Client ---> Router ---> Internet

  • Router runs OpenVPN server
  • Client connects
  • Client uses router DNS, but the router drops the DNS request because it has no interface with that subnet

That about right?

I can add the config setting, but I'm wondering maybe if I should instead add a separate section for this special situation. I'm cautious about this setting because it was expressly added to prevent DNS reflection attacks, which can be very problematic. To me, turning it off is not inconsequential.

@teffalump
that´s right. the wiki says, that the localservice '0' is the default value.
But i would not change the setting within your script.You can not regard all constellations.
Maybe only a hint is helpfull.

@lantis1008
i am not using tor with OpenWRT. So i can´t give you an answer.
But you can just try it if it works for you.

Owner

teffalump commented Oct 9, 2015

@MichiMunich: Yeah, seems that the default is 0... Weird. =-) Anyways, I think, for now, I'll take your suggestion and put a hint about VPN issues if localservice is 1. =-)

Owner

teffalump commented Oct 9, 2015

Just added the hint, although maybe I should stress more caution. Lol.

I run a persistent OpenVPN connection on my router and ad blocking wasn't working at first. Removing the localservice 1 entry from my dnsmasq config worked.

Is there any way to make the ad blocking work without removing the localservice entry? I don't want to open myself up to security issues...

Also, when turning on your script, I get a warning line:

Turning on!
Updating config...
uci: Entry not found
Restarting firewall...
Restarting dnsmasq...
Restarting uhttpd...
Restarting cron...

Is the uci warning normal or expected?

@frontalot
Probably it is because of the command "TOR=uci get tor.global.enabled"
If you don't use TOR you get "uci: Entry not found" as result. So it's not a problem.

when I enable the 'serving with transparent pixel' option then it actually redirects to my openWrt router right? But when it does so, doesn't it interfere with the LUCI login page also listening on port 80 and 443?

Owner

teffalump commented Oct 12, 2015

@frontalot: From what I understand, as long as you take normal precautions, like not listening on external DNS port, then it shouldn't be a problem. But, yeah, I'm not too knowledgeable about this stuff. =-/

Owner

teffalump commented Oct 12, 2015

Pushed change for that Tor check. Shouldn't have that error now. =-)

Owner

teffalump commented Oct 12, 2015

@bonanza123: You are right. =-) Although, what happens is that the redirection has the client request a page from the router, say /advertpage.jsp, and the router cannot find it, so it serves the page_error response (which we've overriden). Consequently, there shouldn't be any problems with the normal operation of the Openwrt web server functionality. However, I have seen some issues. Generally, it's somewhat inelegant and creates subtle issues, but works most of the time. =-)

wget -qO- http://www.mvps.org/winhelp2002/hosts.txt| awk -v r="$ENDPOINT_IP4" '{sub(/^0.0.0.0/, r)} $0 ~ "^"r' > /tmp/block.build.list
wget -qO- "http://adaway.org/hosts.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- http://www.malwaredomainlist.com/hostslist/hosts.txt|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list
#wget -qO- "http://hosts-file.net/.\ad_servers.txt"|awk -v r="$ENDPOINT_IP4" '{sub(/^127.0.0.1/, r)} $0 ~ "^"r' >> /tmp/block.build.list

Thanks for awesome script! Why links 2 and 4 have "", but 1 and 3 are not? It is important?

Owner

teffalump commented Oct 28, 2015

@dartraiden: Link 2 could be unquoted (I believe), but link 4 has a weird, literal backslash that must be kept uninterpreted. Although, it's disabled by default. =-)

uneed commented Oct 29, 2015

very useful,any plan support adblock plus's filters

Owner

teffalump commented Nov 2, 2015

@uneed: Nope. Wouldn't work for a variety of reasons: resource constraints, ssl security, etc. We've talked about this before... somewhere up in the comments. Hehe.

I've noticed that some hosts are added to block.hosts as "0.0.0.0domainnamehere.com", with the space missing. Maybe your awk code needs to add a leading space if the hosts lists contains just domains?

But none of the included lists do not have leading ip's. And looking at my own list I don't observe this behaviour at all.

Owner

teffalump commented Nov 14, 2015

Yeah, like @lantis1008 said, I don't see that behavior @frontalot. There shouldn't be naked domains on the lists. Hmm...no matter, I'll keep an eye out for it. Thanks!

n-land commented Nov 20, 2015

@teffalump: I'd like to try this great script, what is the license? MIT? GPL?

illitum commented Nov 22, 2015

do u create menu position on openwrt to manage this script?

Owner

teffalump commented Nov 29, 2015

@n-land: Say GPL, and we'll go from there. Thanks! =-)

@illitum: No, I need to create the package to have easier management and options, like the menu position. But I've been somewhat lazy in that regard. =-)

remlei commented Dec 12, 2015

it would be cool if I can define a path where to store the blacklist.host file instead of storing it on my router /etc folder and make my router's flash goes bad more quickly.

I use a adblock on tomato but I can define to store the host files on my RAM, which is cool.

Yeah yeah, I know, I can edit the source if I want, anyway I hope you can work on that.

b4hr4m commented Dec 17, 2015

Is this just work on incoming traffic from the WAN port? I'm using my phone to tethering via USB port and I still see the ads.

Owner

teffalump commented Dec 20, 2015

@b4hr4m: I'm having some some trouble understanding what the setup is. The phone is an access point? Or? Basically, if there are unencrypted DNS requests through the router, the ads should be blocked since the router will intercept them. Although there are some unusual cases.

@remlei: Sorry for the delay, but yeah, you have a good point. I should add the ability to define the white and blacklist location. I'll do it when I get back from vacation. In the meantime, yeah, just switch the locations manually. =-)

Thanks for this great plugin. However, wireless only blocking isn't working for me. At first I configured the adblock to block all advertisement, which I later changed to wireless only. And while the script is clearly shows the wireless only option is enabled it still blocks everything on my wired network as well. I've tried reinstalling the script with the -r parameter but that didn't work.

I currently only have two lines in my iptables.

remlei commented Jan 25, 2016

im using mwan3 on openwrt, every time the blocklist updates, mwan3 breaks and need to restarted the service manually.

norman25 commented Feb 1, 2016

i thing the next step to do is creating a Web Interface For The Ad-blocking like the pi-hole web https://github.com/Anon135813/dnsblocker-webgui - http://jacobsalmela.com/a-web-interface-for-the-ad-blocking-pi-hole/ source https://pi-hole.net/

@norman25 my plugin already does this.

norman25 commented Feb 2, 2016

@lantis1008 wow i see it, but i can't use it.
my old router is not supported by gargoyle.

Owner

teffalump commented Feb 27, 2016

@norman25: I'm probably not going to add more to this script, the other plugins are way better now anyways - I would move to those. The openwrt package should work for your router, no? =-)

Owner

teffalump commented Feb 27, 2016

@remlei: Hmm... Yeah, no idea really -- updating is just rereading a new list, don't understand why it would break... I don't know mwan3 that well, although I perused the source. Lol. =-/

fay27 commented May 4, 2017 edited

adaway now redirects to https. script needs to be updated

root@OpenWrt:/# wget http://adaway.org/hosts.txt
--2017-05-04 16:28:17-- http://adaway.org/hosts.txt
Resolving adaway.org... 2400:cb00:2048:1::6818:6959, 104.24.105.89, 104.24.104.89
Connecting to adaway.org|2400:cb00:2048:1::6818:6959|:80... failed: Permission denied.
Connecting to adaway.org|104.24.105.89|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://adaway.org/hosts.txt [following]
--2017-05-04 16:28:18-- https://adaway.org/hosts.txt
Connecting to adaway.org|104.24.105.89|:443... connected.
ERROR: cannot verify adaway.org's certificate, issued by 'CN=COMODO ECC Domain Validation Secure Server CA 2,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB':
Unable to locally verify the issuer's authority.
To connect to adaway.org insecurely, use `--no-check-certificate'.

Hi Guys

I have a problem with OpenVPN + this script. @dcavni It seems I have the same issue so, I'd like to know how you solved it

root@OpenWrt:~# uci get dhcp.@dnsmasq[0].localservice
0
root@OpenWrt:~# uci show | grep openvpn
...
openvpn.sample_server.server='10.8.0.0 255.255.255.0'

OpenVPN works without running this script

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment