Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Pi-hole Youtube ad blocking
#!/bin/sh
# This script will fetch the Googlevideo ad domains and append them to the Pi-hole block list.
# Run this script daily with a cron job (don't forget to chmod +x)
# More info here: https://discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts
# Fetch the list of domains, remove the ip's and save them
curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' \
| awk -F, 'NR>1{print $1}' \
| grep -vE "redirector|manifest" > $FILE
# Replace r*.sn*.googlevideo.com URLs to r*---sn-*.googlevideo.com
# and add those to the list too
cat $FILE | sed -r 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/' >> $FILE
# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE
# Add to Pi-hole adlists if it's not there already
if ! grep $FILE < /etc/pihole/adlists.list; then echo "file://$FILE" >> /etc/pihole/adlists.list; fi;
@prateek33

This comment has been minimized.

Copy link

@prateek33 prateek33 commented Apr 4, 2018

@leeuwtjex

This comment has been minimized.

Copy link

@leeuwtjex leeuwtjex commented Aug 28, 2019

Guys, I have a problem here.
My drive only has 20 gb storage...., the youtube.hosts file eats up all space.
The other day I had to do a trick to find the source of the problem
which was youtube.hosts
What is the solution for that problem? At the pihole forum other people claim they have this situation to.
But the pihole group responds logically 'Sorry, youtube.hosts is not our department...'

Besides that, all respect for this interesting script. Nowadays, I still find youtube adds.
Is there new development to handle 2019?

Thanks for all your effort.

@Kapujino

This comment has been minimized.

Copy link

@Kapujino Kapujino commented Aug 31, 2019

Hey @leeuwtjex,
I had the same problem and fixed it by deleting the old youtube.hosts and editing the sed line to this:
sed -i $FILE -re 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/'
I hope that helps you!

Also had to edit the regex at the wolfram grep since it didn't return anything (according to the website https://regexr.com/ your grep should work though). For me grep -Eo "r\d+---sn-.+.googlevideo.com" didn't find e.g. r1---sn-oguelned.googlevideo.com so I edited the grep to grep -Eo "r.+---sn-.+googlevideo.com" which works.

Thanks for sharing the script, it is working perfectly fine for over a year now.

@leeuwtjex

This comment has been minimized.

Copy link

@leeuwtjex leeuwtjex commented Sep 2, 2019

Thanks Kapujino, I found this mod exactly like you said in another post to in a reddit post somewhere.
It works! I still have 19 gb free now!.

@Pouya-moh

This comment has been minimized.

Copy link

@Pouya-moh Pouya-moh commented Jul 18, 2020

My I ask what does that wolfram API call do? I'm trying to make this happen without registering at wolfram. Thanks.

@ericknorr

This comment has been minimized.

Copy link

@ericknorr ericknorr commented Jul 19, 2020

You need the $APPID$ for Wolfram to work

Replace $APPID$ with your AppId from Wolfram https://developer.wolframalpha.com/portal/signin.html

@Pouya-moh

This comment has been minimized.

Copy link

@Pouya-moh Pouya-moh commented Jul 19, 2020

@ericknorr, Thanks mate, I appreciate. But that was not what I'm asking, I know how to use APIs. My questions was, rather, what is wolfram alpha is doing here. You know, the first API call gets the googlevideo subdomains from hackertarget, then with some awk and sed magic it transforms them to these ---sn like format but then, what is the purpose of call to wolfram services?

@ErikFontanel

This comment has been minimized.

Copy link
Owner Author

@ErikFontanel ErikFontanel commented Jul 20, 2020

@Pouya-moh WA is able to retrieve more domains than hackertarget. But don't bother anymore, as this script is deprecated because YT changed how they serve ads. More info in this thread

@dev-2-4-h

This comment has been minimized.

Copy link

@dev-2-4-h dev-2-4-h commented Sep 11, 2020

This is a long going thread, however the solution (script) works well.
I applied some changes:
#!/bin/sh
# More info here: https:#discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts

# Fetch the list of domains, remove the ip's and save them
curl 'https:#api.hackertarget.com/hostsearch/?q=googlevideo.com' \
| awk -F, 'NR>1{print $1}' \
| grep -vE "redirector|manifest" > $FILE

# Replace r*.sn*.googlevideo.com URLs to r*---sn-*.googlevideo.com
# and add those to the list too

sed -i $FILE -re 's/(^r[[:digit:]]+)(\.)(sn)/\1---\3-/'

# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE

# Update Gravity
pihole -g

I added the list to the list group using the URL: file:///etc/pihole/youtube.hosts
I then edit my cron jobs
sudo crontab -e
adding
*/10 * * * * /etc/pihole/youtube-adblock.sh
This executes the script every 10 minutes, replacing the file (youtube.hosts) with the latest domains.
Works well - I only get Ad's one time, after this they are blocked.

@Nikodermus

This comment has been minimized.

Copy link

@Nikodermus Nikodermus commented Nov 11, 2020

@dev-2-4-h would you mind doing a step by step guide for those who are not particularly familiar with bash scripting? I already have my pi hole in place but I still see the same youtube ads as usual.

I get this error when I try to run the CURL
curl: (6) Could not resolve host: https

@p96xl

This comment has been minimized.

Copy link

@p96xl p96xl commented Dec 8, 2020

@dev-2-4-h would you mind doing a step by step guide for those who are not particularly familiar with bash scripting? I already have my pi hole in place but I still see the same youtube ads as usual.

I get this error when I try to run the CURL
curl: (6) Could not resolve host: https

I just changed "curl 'https:#api.hackertarget.com/hostsearch/?q=googlevideo.com' " to "curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' "

@wdrury-uk

This comment has been minimized.

Copy link

@wdrury-uk wdrury-uk commented Jan 10, 2021

Im running pih-hole v5.2.2 and I seem to be missing something as the AdList is not updating with the YouTube hosts (domains).

I have the script in my /etc/pihole folder as per above revised by @dev-2-4-h last post here

It Successfully creates a youtube.hosts file in the /etc/pihole folder
It Successfull call pihole -g and the update appears to run with success into GravityDB, but no new domains are added.

Looking at the script above I see no obvious way the youtube.hosts file is added to Pi-Hole automatically. Also checked the gravity.sh script Pi-Hole uses when executes the pihole -g command and that doesnt appear to look for .hosts files on updating GravityDB.

Therefore what am I missing, how does pihole -g suppose to add the contents of the auto-generated youtube.hosts file to Pi-Hole?

@rahulmangaldas

This comment has been minimized.

Copy link

@rahulmangaldas rahulmangaldas commented Mar 16, 2021

Cleaned up the script a little bit by replacing the curl/awk/grep/sed sequence with just curl and sed.

#!/bin/sh
# More info here: https:#discourse.pi-hole.net/t/how-do-i-block-ads-on-youtube/253/136
# File to store the YT ad domains
FILE=/etc/pihole/youtube.hosts

# Fetch the list of domains, remove the ip's and save them
curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com' | \
sed -E '1d;/redirector|manifest/d;s/(^r[0-9]+)\.(sn)(.*)(,.*)/\1---\2-\3/' > $FILE

# Scan log file for previously accessed domains
grep '^r.*googlevideo\.com' /var/log/pihole*.log \
| awk '{print $8}' \
| grep -vE "redirector|manifest" \
| sort | uniq >> $FILE

# Update Gravity
pihole -g

@andreinacat

This comment has been minimized.

Copy link

@andreinacat andreinacat commented Mar 17, 2021

maybe this is neccesary to append the youtube domains list to the other already validated, works for me .

bash -c 'cat youtube.hosts >> list.1.raw.githubusercontent.com.domains'

@andreinacat

This comment has been minimized.

Copy link

@andreinacat andreinacat commented Mar 17, 2021

i believe this would be a solution.
`
FILE=/etc/pihole/youtube.hosts

curl 'https://api.hackertarget.com/hostsearch/?q=googlevideo.com'
| awk -F, 'NR>1{print $1}'
| grep -vE "redirector|manifest" > $FILE

sed -i $FILE -re 's/(^r[[:digit:]]+)(.)(sn)/\1---\3-/'

grep '^r.googlevideo.com' /var/log/pihole.log
| awk '{print $8}'
| grep -vE "redirector|manifest"
| sort | uniq >> $FILE

bash -c 'cat /etc/pihole/youtube.hosts >> list.1.raw.githubusercontent.com.domains'

pihole -g`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment