Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Pi-hole Youtube ad blocking
#!/bin/sh
# This script will fetch the Googlevideo ad domains and append them to the Pi-hole block list.
# Run this script daily with a cron job (don't forget to chmod +x)
# More info here: https://discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts
# Fetch the list of domains, remove the ip's and save them
curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' \
| awk -F, 'NR>1{print $1}' \
| grep -vE "redirector|manifest" > $FILE
# Replace r*.sn*.googlevideo.com URLs to r*---sn-*.googlevideo.com
# and add those to the list too
cat $FILE | sed -r 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/' >> $FILE
# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE
# Add to Pi-hole adlists if it's not there already
if ! grep $FILE < /etc/pihole/adlists.list; then echo "file://$FILE" >> /etc/pihole/adlists.list; fi;
@prateek33
Copy link

prateek33 commented Apr 4, 2018

@leeuwtjex
Copy link

leeuwtjex commented Aug 28, 2019

Guys, I have a problem here.
My drive only has 20 gb storage...., the youtube.hosts file eats up all space.
The other day I had to do a trick to find the source of the problem
which was youtube.hosts
What is the solution for that problem? At the pihole forum other people claim they have this situation to.
But the pihole group responds logically 'Sorry, youtube.hosts is not our department...'

Besides that, all respect for this interesting script. Nowadays, I still find youtube adds.
Is there new development to handle 2019?

Thanks for all your effort.

@Kapujino
Copy link

Kapujino commented Aug 31, 2019

Hey @leeuwtjex,
I had the same problem and fixed it by deleting the old youtube.hosts and editing the sed line to this:
sed -i $FILE -re 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/'
I hope that helps you!

Also had to edit the regex at the wolfram grep since it didn't return anything (according to the website https://regexr.com/ your grep should work though). For me grep -Eo "r\d+---sn-.+.googlevideo.com" didn't find e.g. r1---sn-oguelned.googlevideo.com so I edited the grep to grep -Eo "r.+---sn-.+googlevideo.com" which works.

Thanks for sharing the script, it is working perfectly fine for over a year now.

@leeuwtjex
Copy link

leeuwtjex commented Sep 2, 2019

Thanks Kapujino, I found this mod exactly like you said in another post to in a reddit post somewhere.
It works! I still have 19 gb free now!.

@Pouya-moh
Copy link

Pouya-moh commented Jul 18, 2020

My I ask what does that wolfram API call do? I'm trying to make this happen without registering at wolfram. Thanks.

@ericknorr
Copy link

ericknorr commented Jul 19, 2020

You need the $APPID$ for Wolfram to work

Replace $APPID$ with your AppId from Wolfram https://developer.wolframalpha.com/portal/signin.html

@Pouya-moh
Copy link

Pouya-moh commented Jul 19, 2020

@ericknorr, Thanks mate, I appreciate. But that was not what I'm asking, I know how to use APIs. My questions was, rather, what is wolfram alpha is doing here. You know, the first API call gets the googlevideo subdomains from hackertarget, then with some awk and sed magic it transforms them to these ---sn like format but then, what is the purpose of call to wolfram services?

@ErikFontanel
Copy link
Author

ErikFontanel commented Jul 20, 2020

@Pouya-moh WA is able to retrieve more domains than hackertarget. But don't bother anymore, as this script is deprecated because YT changed how they serve ads. More info in this thread

@dev-2-4-h
Copy link

dev-2-4-h commented Sep 11, 2020

This is a long going thread, however the solution (script) works well.
I applied some changes:
#!/bin/sh
# More info here: https:#discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts

# Fetch the list of domains, remove the ip's and save them
curl 'https:#api.hackertarget.com/hostsearch/?q=googlevideo.com' \
| awk -F, 'NR>1{print $1}' \
| grep -vE "redirector|manifest" > $FILE

# Replace r*.sn*.googlevideo.com URLs to r*---sn-*.googlevideo.com
# and add those to the list too

sed -i $FILE -re 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/'

# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE

# Update Gravity
pihole -g

I added the list to the list group using the URL: file:///etc/pihole/youtube.hosts
I then edit my cron jobs
sudo crontab -e
adding
*/10 * * * * /etc/pihole/youtube-adblock.sh
This executes the script every 10 minutes, replacing the file (youtube.hosts) with the latest domains.
Works well - I only get Ad's one time, after this they are blocked.

@Nikodermus
Copy link

Nikodermus commented Nov 11, 2020

@dev-2-4-h would you mind doing a step by step guide for those who are not particularly familiar with bash scripting? I already have my pi hole in place but I still see the same youtube ads as usual.

I get this error when I try to run the CURL
curl: (6) Could not resolve host: https

@p96xl
Copy link

p96xl commented Dec 8, 2020

@dev-2-4-h would you mind doing a step by step guide for those who are not particularly familiar with bash scripting? I already have my pi hole in place but I still see the same youtube ads as usual.

I get this error when I try to run the CURL
curl: (6) Could not resolve host: https

I just changed "curl 'https:#api.hackertarget.com/hostsearch/?q=googlevideo.com' " to "curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' "

@wdrury-uk
Copy link

wdrury-uk commented Jan 10, 2021

Im running pih-hole v5.2.2 and I seem to be missing something as the AdList is not updating with the YouTube hosts (domains).

I have the script in my /etc/pihole folder as per above revised by @dev-2-4-h last post here

It Successfully creates a youtube.hosts file in the /etc/pihole folder
It Successfull call pihole -g and the update appears to run with success into GravityDB, but no new domains are added.

Looking at the script above I see no obvious way the youtube.hosts file is added to Pi-Hole automatically. Also checked the gravity.sh script Pi-Hole uses when executes the pihole -g command and that doesnt appear to look for .hosts files on updating GravityDB.

Therefore what am I missing, how does pihole -g suppose to add the contents of the auto-generated youtube.hosts file to Pi-Hole?

@rahulmangaldas
Copy link

rahulmangaldas commented Mar 16, 2021

Cleaned up the script a little bit by replacing the curl/awk/grep/sed sequence with just curl and sed.

#!/bin/sh
# More info here: https:#discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts

# Fetch the list of domains, remove the ip's and save them
curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' | \
sed -E '1d;/redirector|manifest/d;s/(^r[0-9]+)\.(sn)(.*)(,.*)/\1---\2-\3/' > $FILE

# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE

# Update Gravity
pihole -g

@andreinacat
Copy link

andreinacat commented Mar 17, 2021

maybe this is neccesary to append the youtube domains list to the other already validated, works for me .

bash -c 'cat youtube.hosts >> list.1.raw.githubusercontent.com.domains'

@andreinacat
Copy link

andreinacat commented Mar 17, 2021

i believe this would be a solution.
`
FILE=/etc/pihole/youtube.hosts

curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com'
| awk -F, 'NR>1{print $1}'
| grep -vE "redirector|manifest" > $FILE

sed -i $FILE -re 's/(^r[[:digit:]]+)(.)(sn)/\1---\3-/'

grep '^r.googlevideo.com' /var/log/pihole.log
| awk '{print $8}'
| grep -vE "redirector|manifest"
| sort | uniq >> $FILE

bash -c 'cat /etc/pihole/youtube.hosts >> list.1.raw.githubusercontent.com.domains'

pihole -g`

@rjhancock
Copy link

rjhancock commented Feb 13, 2022

An alternative may be to add the following regex on the blacklist:

^r.+\.googlevideo.com$

Seems to work for me locally. The ad segments still appear but it's just a spinner for 5 seconds then 'Skip Ads' appear.

@leeuwtjex
Copy link

leeuwtjex commented Feb 14, 2022

rjhancock,

Thanks for the idea, I just tried it cause I didn't understand it. After adding the regex ^r.+.googlevideo.com$
all youtube movies are blocked....not one runs.
How local did you mean?

@rjhancock
Copy link

rjhancock commented Feb 14, 2022

Yea, I removed it last night as it looks like about half the videos are blocked if not more. Bad idea. So I wonder if there is a different pattern that might work instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment