Skip to content

Instantly share code, notes, and snippets.

@jult
Last active March 5, 2024 07:33
Show Gist options
  • Star 77 You must be signed in to star a gist
  • Fork 10 You must be signed in to fork a gist
  • Save jult/e76c628899bd5aa3c33a to your computer and use it in GitHub Desktop.
Save jult/e76c628899bd5aa3c33a to your computer and use it in GitHub Desktop.
blocking p2p peer snoopers and evil corp (for Tixati IP filter and such)
#!/bin/sh
# This script runs every other night at 04:56 CET on a webserver I maintain
# Results are always at: https://jult.net/block.txt ( or https://jult.net/block.txt.gz )
# And much smaller, stripped of BS; https://jult.net/bloc.txt
# For use in Tixati IP filter: https://jult.net/bloc.txt.gz !!!
# And finally a txt file with just the bold IP-ranges: https://jult.net/bl.txt (or https://jult.net/bl.txt.gz )
# Download open block-lists, unpack, filter:
curl -s https://www.iblocklist.com/lists.php | grep -A 2 Bluetack | xargs wget -qO - --limit-rate=500k | gunzip -f | egrep -v '^#' > /tmp/xbp
sleep 9
# Assholes allowing kiddy-porn, uncomment if you are prepared to deprive yourself of speedy (non-porn) downloads as well, for a good cause:
#cd /tmp
#wget -q --trust-server-names http://list.iblocklist.com/?list=dufcxgnbjsdwmwctgfuj&fileformat=p2p&archiveformat=gz
#sleep 4
#cd /tmp
#gunzip duf*.gz
#cat /tmp/duf* >> /tmp/xbp
# and another list:
wget -qO - --limit-rate=500k http://www.wael.name/wael.list.p2p.gz > /tmp/xb2.gz
gunzip -f /tmp/xb2.gz
cat /tmp/xb2 >> /tmp/xbp
sleep 9
# and another
wget -qO - --limit-rate=500k http://john.bitsurge.net/public/biglist.p2p.gz > /tmp/xb3.gz
gunzip -f /tmp/xb3.gz
cat /tmp/xb3 >> /tmp/xbp
sleep 9
# Strip, Sort and Delete doubles:
sed -i "s/[[:space:]]*#.*$//g" /tmp/xbp
sleep 9
sed -i "s/.*value='\(http:.*\)'.*/\1/p" /tmp/xbp
sleep 9
sort /tmp/xbp | uniq > /yourwebdocroot/block.txt
sleep 9
# Strip everything but IPv4s (and ranges), for use in firewalls etc.
grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}$|([0-9]{1,3}\.){3}[0-9]{1,3}\-([0-9]{1,3}\.){3}[0-9]{1,3}' /yourwebdocroot/block.txt > /tmp/xbl
sleep 9
sort /tmp/xbl | uniq > /yourwebdocroot/bl.txt
sleep 9
# Beautyfi for tixati pulling, add colon in front
sed "s/^/:/g" /yourwebdocroot/bl.txt > /yourwebdocroot/bloc.txt
# Keep as txt and gz file:
gzip -c /yourwebdocroot/block.txt > /yourwebdocroot/block.txt.gz
sleep 9
gzip -c /yourwebdocroot/bloc.txt > /yourwebdocroot/bloc.txt.gz
sleep 9
gzip -c /yourwebdocroot/bl.txt > /yourwebdocroot/bl.txt.gz
sleep 9
# Let's make sure next run is a clean one
rm -rf /tmp/xb*
#rm -rf /tmp/du*
exit 0
@jult
Copy link
Author

jult commented Feb 29, 2016

For the minimalist systems; It requires cat, comm, curl, wget. gzip, gunzip, rm, sed, sort and uniq

@mohamedation
Copy link

thank you!

@Marucins
Copy link

Are You updating these lists? (check for errors?)

@jult
Copy link
Author

jult commented Nov 14, 2016

I have no access to the source lists, no, but I do check my parsing for errors often..

@Kareltje1980
Copy link

Hi, is this still working?

@kolonuk
Copy link

kolonuk commented Nov 5, 2017

Whats the deal with all the "sleep 9"s?

@ohsevin
Copy link

ohsevin commented Apr 7, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment