-
-
Save johntyree/3331662 to your computer and use it in GitHub Desktop.
| #!/usr/bin/env sh | |
| # Download lists, unpack and filter, write to stdout | |
| curl -s https://www.iblocklist.com/lists.php \ | |
| | sed -n "s/.*value='\(http:.*=bt_.*\)'.*/\1/p" \ | |
| | xargs wget -O - \ | |
| | gunzip \ | |
| | egrep -v '^#' |
You are redistributing iblocklist.com data without permission. Please take down this file;Should I just ignore them?
No, you shouldn't. You should take it down. They asked you kindly not to redistribute data they provide without their permission. They have every right to do that.
That's why people like myself opt to create and provide scraper apps/scripts instead of distributing actual data.
That's why people like myself opt to create and provide scraper apps/scripts instead of distributing actual data.
What's interesting, there are plenty of repos on GitHub containing same list files, yet those are up longer than my private mirror does. And the free lists they distribute can be sourced from other sites than iblocklist.com, so I'm more curious whether other owners have received such an email.
What's interesting, there are plenty of repos on GitHub containing same list files, yet those are up longer than my private mirror does. And the free lists they distribute can be sourced from other sites than iblocklist.com, so I'm more curious whether other owners have received such an email.
Well, might just be coincidence, might be that the others received takedown notices too. You'll probably never know. Doesn't change the fact that they contacted you and expect you to act, though.
@fonic I just wrote them to go away and pulled off one of their lists that my script was downloading, but it's still not ok how they communicate because instead of writing me directly what the problem is, I had to figure it out on my own. However, what annoys me the most is that they ignored my questions and threatened to reveal my name and answers to their emails on their site, very unprofessional.
Oh, how ironic...
What site is that screenshot from?
just warm remind to all here's message from iblocklist to me they asked shutdown my github repo and here's why:
We don't mind sharing scripts that access our service.
We don't want the list data to be redistributed.
I hope all of you if redistributed now, shutdown it now, script are fine I think.
@sahsu they can go somewhere else, they don't even own the data except their one free list.
They even threatened me without explaining why then other websites like firehol.org actually provide a copy to be re-distributed...
Pretty soon http://john.bitsurge.net/public/biglist.p2p.gz is likely to disappear anyway because my host is discontinuing the service plan that made hosting this file affordable. Even now, with it being way past its prime, it's still quite a lot of network egress per month.
Even if my hosting costs weren't changing, as this script gets more and more out of date it becomes less and less responsible for me to keep letting people rely on it. Maybe if their clients start failing to download they will update their configs to point to something more recent. There are lots of alternatives.
@johntyree
hi
sadly now the link is down , is there a mirror?
thanks
Pretty soon http://john.bitsurge.net/public/biglist.p2p.gz is likely to disappear anyway because my host is discontinuing the service plan that made hosting this file affordable. Even now, with it being way past its prime, it's still quite a lot of network egress per month.
Even if my hosting costs weren't changing, as this script gets more and more out of date it becomes less and less responsible for me to keep letting people rely on it. Maybe if their clients start failing to download they will update their configs to point to something more recent. There are lots of alternatives.@johntyree
hi
sadly now the link is down , is there a mirror?
thanks
Use iplists.firehol.org or build the list yourself. I actually provide a iblocklist free list on my mirror: https://mirror.codebucket.de/transmission/
Dear everyone,
The file is no longer hosted at john.bitsurge.net and never will be again. It languished unmaintained for a very long time anyway. My suggestion to you is to run the script(s) yourself now and then. Anyone hosting iblocklists lists will eventually be asked to stop.
Thanks John for your efforts all these years. All the best.
. It languished unmaintained for a very long time anyway. My suggestion to you is to run the script(s) yourself now and then. Anyone hosting iblocklists lists will eventually be asked to stop.
@johntyree
Hi
may i ask you a question ?
may i know which filters have you used in the past to build your biglist.p2p.gz ?
thanks
Dear everyone,
The file is no longer hosted at john.bitsurge.net and never will be again. It languished unmaintained for a very long time anyway. My suggestion to you is to run the script(s) yourself now and then. Anyone hosting iblocklists lists will eventually be asked to stop.
Thanks.
may i ask you a question ?
you may not. john has left.
This thread should be locked.
curl -A "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/81.0" -s https://www.iblocklist.com/lists.php \
| sed -n "s/.*value='\(http:.*\)'.*/\1/p" \
| xargs wget -O - \
| gunzip \
| egrep -v '^#'
Funny enough I got now a takedown request from iblocklist.com as I have a copy of the biglist on my mirror, even though those lists aren't even authored by them...
Should I just ignore them?