Skip to content

Instantly share code, notes, and snippets.

@johntyree
Last active March 9, 2024 12:32
Show Gist options
  • Save johntyree/3331662 to your computer and use it in GitHub Desktop.
Save johntyree/3331662 to your computer and use it in GitHub Desktop.
Make one large blocklist from the bluetack lists on iblocklist.com
#!/usr/bin/env sh
# Download lists, unpack and filter, write to stdout
curl -s https://www.iblocklist.com/lists.php \
| sed -n "s/.*value='\(http:.*=bt_.*\)'.*/\1/p" \
| xargs wget -O - \
| gunzip \
| egrep -v '^#'
@sahsu
Copy link

sahsu commented Jul 1, 2021

just warm remind to all here's message from iblocklist to me they asked shutdown my github repo and here's why:

We don't mind sharing scripts that access our service.
We don't want the list data to be redistributed.

I hope all of you if redistributed now, shutdown it now, script are fine I think.

@divadsn
Copy link

divadsn commented Jul 1, 2021

@sahsu they can go somewhere else, they don't even own the data except their one free list.

They even threatened me without explaining why then other websites like firehol.org actually provide a copy to be re-distributed...

@mantrasiva
Copy link

Pretty soon http://john.bitsurge.net/public/biglist.p2p.gz is likely to disappear anyway because my host is discontinuing the service plan that made hosting this file affordable. Even now, with it being way past its prime, it's still quite a lot of network egress per month.

Even if my hosting costs weren't changing, as this script gets more and more out of date it becomes less and less responsible for me to keep letting people rely on it. Maybe if their clients start failing to download they will update their configs to point to something more recent. There are lots of alternatives.

@johntyree
hi
sadly now the link is down , is there a mirror?
thanks

@divadsn
Copy link

divadsn commented Jul 2, 2021

Pretty soon http://john.bitsurge.net/public/biglist.p2p.gz is likely to disappear anyway because my host is discontinuing the service plan that made hosting this file affordable. Even now, with it being way past its prime, it's still quite a lot of network egress per month.
Even if my hosting costs weren't changing, as this script gets more and more out of date it becomes less and less responsible for me to keep letting people rely on it. Maybe if their clients start failing to download they will update their configs to point to something more recent. There are lots of alternatives.

@johntyree
hi
sadly now the link is down , is there a mirror?
thanks

Use iplists.firehol.org or build the list yourself. I actually provide a iblocklist free list on my mirror: https://mirror.codebucket.de/transmission/

@johntyree
Copy link
Author

Dear everyone,

The file is no longer hosted at john.bitsurge.net and never will be again. It languished unmaintained for a very long time anyway. My suggestion to you is to run the script(s) yourself now and then. Anyone hosting iblocklists lists will eventually be asked to stop.

@YesTrustMe
Copy link

Thanks John for your efforts all these years. All the best.

@mantrasiva
Copy link

. It languished unmaintained for a very long time anyway. My suggestion to you is to run the script(s) yourself now and then. Anyone hosting iblocklists lists will eventually be asked to stop.

@johntyree
Hi
may i ask you a question ?
may i know which filters have you used in the past to build your biglist.p2p.gz ?
thanks

@mrglutton
Copy link

Dear everyone,

The file is no longer hosted at john.bitsurge.net and never will be again. It languished unmaintained for a very long time anyway. My suggestion to you is to run the script(s) yourself now and then. Anyone hosting iblocklists lists will eventually be asked to stop.

Thanks.

@YesTrustMe
Copy link

may i ask you a question ?

you may not. john has left.

@YesTrustMe
Copy link

This thread should be locked.

@zfkdnop
Copy link

zfkdnop commented Aug 1, 2021

curl -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/81.0" -s https://www.iblocklist.com/lists.php \
 | sed -n "s/.*value='\(http:.*\)'.*/\1/p" \
 | xargs wget -O - \
 | gunzip \
 | egrep -v '^#'

@johntyree
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment