Skip to content

Instantly share code, notes, and snippets.

@ErikFontanel
Last active November 28, 2023 18:55
Show Gist options
  • Save ErikFontanel/4ee1ab393b119690a293ba558976b113 to your computer and use it in GitHub Desktop.
Save ErikFontanel/4ee1ab393b119690a293ba558976b113 to your computer and use it in GitHub Desktop.
Pi-hole Youtube ad blocking
#!/bin/sh
# This script will fetch the Googlevideo ad domains and append them to the Pi-hole block list.
# Run this script daily with a cron job (don't forget to chmod +x)
# More info here: https://discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts
# Fetch the list of domains, remove the ip's and save them
curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' \
| awk -F, 'NR>1{print $1}' \
| grep -vE "redirector|manifest" > $FILE
# Replace r*.sn*.googlevideo.com URLs to r*---sn-*.googlevideo.com
# and add those to the list too
cat $FILE | sed -r 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/' >> $FILE
# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE
# Add to Pi-hole adlists if it's not there already
if ! grep $FILE < /etc/pihole/adlists.list; then echo "file://$FILE" >> /etc/pihole/adlists.list; fi;
@Pouya-moh
Copy link

My I ask what does that wolfram API call do? I'm trying to make this happen without registering at wolfram. Thanks.

@ericknorr
Copy link

You need the $APPID$ for Wolfram to work

Replace $APPID$ with your AppId from Wolfram https://developer.wolframalpha.com/portal/signin.html

@Pouya-moh
Copy link

@ericknorr, Thanks mate, I appreciate. But that was not what I'm asking, I know how to use APIs. My questions was, rather, what is wolfram alpha is doing here. You know, the first API call gets the googlevideo subdomains from hackertarget, then with some awk and sed magic it transforms them to these ---sn like format but then, what is the purpose of call to wolfram services?

@ErikFontanel
Copy link
Author

@Pouya-moh WA is able to retrieve more domains than hackertarget. But don't bother anymore, as this script is deprecated because YT changed how they serve ads. More info in this thread

@dev-2-4-h
Copy link

This is a long going thread, however the solution (script) works well.
I applied some changes:
#!/bin/sh
# More info here: https:#discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts

# Fetch the list of domains, remove the ip's and save them
curl 'https:#api.hackertarget.com/hostsearch/?q=googlevideo.com' \
| awk -F, 'NR>1{print $1}' \
| grep -vE "redirector|manifest" > $FILE

# Replace r*.sn*.googlevideo.com URLs to r*---sn-*.googlevideo.com
# and add those to the list too

sed -i $FILE -re 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/'

# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE

# Update Gravity
pihole -g

I added the list to the list group using the URL: file:///etc/pihole/youtube.hosts
I then edit my cron jobs
sudo crontab -e
adding
*/10 * * * * /etc/pihole/youtube-adblock.sh
This executes the script every 10 minutes, replacing the file (youtube.hosts) with the latest domains.
Works well - I only get Ad's one time, after this they are blocked.

@Nikodermus
Copy link

Nikodermus commented Nov 11, 2020

@dev-2-4-h would you mind doing a step by step guide for those who are not particularly familiar with bash scripting? I already have my pi hole in place but I still see the same youtube ads as usual.

I get this error when I try to run the CURL
curl: (6) Could not resolve host: https

@p96xl
Copy link

p96xl commented Dec 8, 2020

@dev-2-4-h would you mind doing a step by step guide for those who are not particularly familiar with bash scripting? I already have my pi hole in place but I still see the same youtube ads as usual.

I get this error when I try to run the CURL
curl: (6) Could not resolve host: https

I just changed "curl 'https:#api.hackertarget.com/hostsearch/?q=googlevideo.com' " to "curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' "

@wdrury-uk
Copy link

wdrury-uk commented Jan 10, 2021

Im running pih-hole v5.2.2 and I seem to be missing something as the AdList is not updating with the YouTube hosts (domains).

I have the script in my /etc/pihole folder as per above revised by @dev-2-4-h last post here

It Successfully creates a youtube.hosts file in the /etc/pihole folder
It Successfull call pihole -g and the update appears to run with success into GravityDB, but no new domains are added.

Looking at the script above I see no obvious way the youtube.hosts file is added to Pi-Hole automatically. Also checked the gravity.sh script Pi-Hole uses when executes the pihole -g command and that doesnt appear to look for .hosts files on updating GravityDB.

Therefore what am I missing, how does pihole -g suppose to add the contents of the auto-generated youtube.hosts file to Pi-Hole?

@conuaunoc
Copy link

conuaunoc commented Mar 16, 2021

Cleaned up the script a little bit by replacing the curl/awk/grep/sed sequence with just curl and sed.

#!/bin/sh
# More info here: https:#discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts

# Fetch the list of domains, remove the ip's and save them
curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' | \
sed -E '1d;/redirector|manifest/d;s/(^r[0-9]+)\.(sn)(.*)(,.*)/\1---\2-\3/' > $FILE

# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE

# Update Gravity
pihole -g

@andreinacat
Copy link

maybe this is neccesary to append the youtube domains list to the other already validated, works for me .

bash -c 'cat youtube.hosts >> list.1.raw.githubusercontent.com.domains'

@andreinacat
Copy link

andreinacat commented Mar 17, 2021

i believe this would be a solution.
`
FILE=/etc/pihole/youtube.hosts

curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com'
| awk -F, 'NR>1{print $1}'
| grep -vE "redirector|manifest" > $FILE

sed -i $FILE -re 's/(^r[[:digit:]]+)(.)(sn)/\1---\3-/'

grep '^r.googlevideo.com' /var/log/pihole.log
| awk '{print $8}'
| grep -vE "redirector|manifest"
| sort | uniq >> $FILE

bash -c 'cat /etc/pihole/youtube.hosts >> list.1.raw.githubusercontent.com.domains'

pihole -g`

@rjhancock
Copy link

An alternative may be to add the following regex on the blacklist:

^r.+\.googlevideo.com$

Seems to work for me locally. The ad segments still appear but it's just a spinner for 5 seconds then 'Skip Ads' appear.

@leeuwtjex
Copy link

rjhancock,

Thanks for the idea, I just tried it cause I didn't understand it. After adding the regex ^r.+.googlevideo.com$
all youtube movies are blocked....not one runs.
How local did you mean?

@rjhancock
Copy link

Yea, I removed it last night as it looks like about half the videos are blocked if not more. Bad idea. So I wonder if there is a different pattern that might work instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment