Skip to content

Instantly share code, notes, and snippets.

@glaszig
Last active January 30, 2022 00:30
Show Gist options
  • Star 31 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
  • Save glaszig/bf96beccf4694ae25d4f1f7cc6224985 to your computer and use it in GitHub Desktop.
Save glaszig/bf96beccf4694ae25d4f1f7cc6224985 to your computer and use it in GitHub Desktop.
creates a blocklist for your p2p endeavors
#!/usr/bin/env sh
# DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
# Version 2, December 2004
#
# Copyright (C) 2020 glaszig <glaszig@gmail.com>
#
# Everyone is permitted to copy and distribute verbatim or modified
# copies of this license document, and changing it is allowed as long
# as the name is changed.
#
# DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
# TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
#
# 0. You just DO WHAT THE FUCK YOU WANT TO.
# inspired by https://gist.github.com/johntyree/3331662#gistcomment-1968023
#
# usage:
# UA="user agent string" mkp2pblocklist > /tmp/blockp2p.list
# or
# UA="user agent string" mkp2pblocklist | gzip -9 > /tmp/blockp2p.list.gz
PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:$PATH"
if [ -z "$UA" ]; then
echo "User Agent environment variable UA missing."
echo "See, e.g., https://user-agents.net/browsers for examples."
exit 1
fi
curl -A "$UA" -s https://www.iblocklist.com/lists.php \
| sed -n "s/.*value='\(http:.*=p2p.*\)'.*/\1/p" \
| xargs curl -A "$UA" -sL \
| gunzip \
| egrep -v '^#' \
| uniq
@GreatWizard
Copy link

GreatWizard commented Sep 22, 2021

For whatever reason @glaszig file contains the following:

The Scary IPv4 Internets:0.0.0.0-255.255.255.255
The Scary IPv6 Internets:0000:0000:0000:0000:0000:0000:0000:0000-FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

I think iblocklist detects its own user agent and returns fake data.

I made an alternative in a GitHub repository.

@glaszig
Copy link
Author

glaszig commented Sep 22, 2021

I made an alternative in a GitHub repository.

looked at it. did you change something? the ua you’re using looks pretty normal.

@GreatWizard
Copy link

GreatWizard commented Nov 9, 2021

I'm quite sure they blacklist your UA.
Do you try to refresh to a new one?

edit: I'm now getting a random UA user-agents.net, that will be more future proof ;)

curl 'https://user-agents.net/download' -X POST --data-raw 'download=txt&browser_bits=64&brower_type=browser&limit=100' > ua.txt
cat ua.txt | sort -R | head -1

@glaszig
Copy link
Author

glaszig commented Jan 30, 2022

iblocklist has become a bullshit operation with their automation and anti-scrapgin policies. is anybody aware of a better source?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment