Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Make one large blocklist from the bluetack lists on iblocklist.com
#!/usr/bin/env sh
# Download lists, unpack and filter, write to stdout
curl -s https://www.iblocklist.com/lists.php \
| sed -n "s/.*value='\(http:.*=bt_.*\)'.*/\1/p" \
| xargs wget -O - \
| gunzip \
| egrep -v '^#'
@studgeek

This comment has been minimized.

Copy link

studgeek commented Sep 22, 2012

This can easily be added to cron also:
0 3 * * 0 curl -s http://www.iblocklist.com/lists.php | sed -n "s/.*value='\(http:.*=bt_.*\)'.*/\1/p" | xargs wget -O - | gunzip | egrep -v '^#' > ~/Library/Application\ Support/Transmission/blocklists/generated.txt

@ArtemGordinsky

This comment has been minimized.

Copy link

ArtemGordinsky commented Oct 11, 2012

Hi! I'm getting these errors trying to execute from the Terminal:
"xargs: wget: No such file or directory
gzip: stdin: unexpected end of file"

Am I doing anything wrong? Thanks!

@krono

This comment has been minimized.

Copy link

krono commented Oct 11, 2012

replace wget -O - by curl in the script.

@panosmdma

This comment has been minimized.

Copy link

panosmdma commented Oct 26, 2012

Hi and thanks for script. I run from terminal (i have replace wget -0 - with curl) and all is:
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
And then error:
gzip: stdin: unexpected end of file
Thanks!

@jamesstout

This comment has been minimized.

Copy link

jamesstout commented Dec 7, 2012

Replace wget -O - with curl -Ls

@ArtemGordinsky

This comment has been minimized.

Copy link

ArtemGordinsky commented Dec 11, 2012

Hi @jamesstout,
Would you please post the entire terminal command with your correction?
When I run:

sudo 0 3 * * 0 curl -s http://www.iblocklist.com/lists.php | sed -n "s/.*value='\(http:.*=bt_.*\)'.*/\1/p" | xargs curl -Ls | gunzip | egrep -v    '^#' > ~/Library/Application\ Support/Transmission/blocklists/generated.txt

I get:

-bash: 0: command not found

@prehensilecode

This comment has been minimized.

Copy link

prehensilecode commented Feb 22, 2013

The "0 3 * * 0 ..." is a crontab entry. Just type "curl -s ...."

@tlongren

This comment has been minimized.

Copy link

tlongren commented May 19, 2013

@ArtemGordinsky you're getting -bash: 0: command not found because 0 isn't a command. It's a crontab entry, as @prehensilecode mentioned.

@fortran01

This comment has been minimized.

Copy link

fortran01 commented Jan 12, 2014

Updated for Mac:

curl -s https://www.iblocklist.com/lists.php | sed -n "s/.*value='\(http:.*=bt_.*\)'.*/\1/p" | sed "s/\&/\&/g" | sed "s/http/\"http/g" | sed "s/gz/gz\"/g" | xargs curl -L | gunzip | egrep -v '^#' > ~/Library/Application\ Support/Transmission/blocklists/generated.txt.bin
@kilolima

This comment has been minimized.

Copy link

kilolima commented Jan 16, 2014

The blocklist URL has changed to HTTPS. The updated url on line 1 is https://www.iblocklist.com/lists.php (as above but it wasn't updated in the download). Thanks for this great script!

@GithubIsAgeist

This comment has been minimized.

Copy link

GithubIsAgeist commented Jan 24, 2014

iblocklist.com is bigoted hate website.

@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Jan 28, 2014

@kilolima updated.

@hedgehoggski

This comment has been minimized.

Copy link

hedgehoggski commented Mar 18, 2014

Hi folks. I've analysed the code for @fortran01's mac version (thanks v. much @fortran01)
and posted a plain english translation of what each bit does below.
Hope it's accurate but if I have made any errors please correct me! :-)
Hope this helps people tweak the code to do other similar stuff

For more clarity as to what's going on, I've put each piped segment of the command on seperate lines,
but remember it's all one big line.

 curl -s https://www.iblocklist.com/lists.php
 | sed -n "s/.*value='\(http:.*=bt_.*\)'.*/\1/p"
 | sed "s/\&/\&/g"
 | sed "s/http/\"http/g"
 | sed "s/gz/gz\"/g"
 | xargs curl -L
 | gunzip
 | egrep -v '^#' > ~/Library/Application\ Support/Transmission/blocklists/generated.txt.bin

Plain english Explanation of each bit of the command does:

grab the webpage "https://www.iblocklist.com/lists.php" in silent mode (no progress bar or error messages)

search each line of this webpage, looking for lines containing text of the form
(anything)value=http:(anything)=bt_(anything)
chop out and dump the first bit ( (anything)value= ) from each of the lines you find

in the resultant lines, change all occurrences of &amp to &

in the resultant lines change all occurrences of the string http to "http

in the resultant lines, change all occurrences of the string gz to gz"

feed the resultant lines one by one to the curl command (-L means curl will automatically redo the grab if server says any file resource has moved)

feed each file downloaded by curl to gunzip program (uncompress it)

write only the lines from each file that don't start with a # (i.e. that are not comments) into the file
"~/Library/Application Support/Transmission/blocklists/generated.txt.bin"

All this ultimately results in the following lines being fed one by one by xargs to the curl command:

"http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_level2&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_edu&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_rangetest&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_bogon&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_ads&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_spyware&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_proxy&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_templist&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_microsoft&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_spider&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_hijacked&fileformat=p2p&archiveformat=gz"
"http://list.iblocklist.com/?list=bt_dshield&fileformat=p2p&archiveformat=gz"

This should help clarify exactly what pattern is being searched for by sed and what the sed filtering actually does
e.g.
"&amp" becomes "&"
inverted commas placed before each http and after each gz

You could of course feed these lines manually to curl if you just want to grab individual zipped blocklists etc.

If doing this, don't forget to filter out the comment lines with egrep and write to a .bin file for transmission :-)

@ronnicek

This comment has been minimized.

Copy link

ronnicek commented May 26, 2014

aww.. they changed it again :/

@flowolf

This comment has been minimized.

Copy link

flowolf commented Jun 2, 2014

@Piezoid

This comment has been minimized.

Copy link

Piezoid commented Jun 4, 2014

I've just wrote a simple Haskell program to sort and merge IP ranges (overlapping and adjacent IP ranges are fused, empty or commented lines are removed) : https://gist.github.com/Piezoid/ee43be6e5eebd6aa9bac

Use stdin and stdout with gz format :
./fuseblkl < inList.p2p.gz > outList.p2p.gz

@BBcan177

This comment has been minimized.

Copy link

BBcan177 commented Jun 24, 2014

I wrote a script that downloads 50 different Blocklists and performs Duplication checks and other Repeated Offender Queries to compact the IP Reputation from a Threat Source of over 700,000 IPs to approx 2-300,000 IPs. Its primarily for pfSense Firewall Blocking but can be adapted for other needs. Any comments appreciated via email.

https://gist.github.com/BBcan17/67e8c456cb399fbe02ee

@rwilhelm

This comment has been minimized.

Copy link

rwilhelm commented Jul 28, 2014

Here's another variant: it downloads only lists with a rating of >= 4. You could remove grep -A1 'star_[45]' to get all lists.

curl -sL 'https://www.iblocklist.com/lists.php' | egrep -A1 'star_[45]' | egrep -o '[a-z]{20}' | sort -u | while read -r blocklist; do curl -sL "http://list.iblocklist.com/?list=${blocklist}&fileformat=p2p&archiveformat=gz" | gunzip -q > ~/Library/Application\ Support/Transmission/blocklists/$blocklist; done

https://trac.transmissionbt.com/wiki/Blocklists:

When transmission starts, it scans this directory for files not ending in ".bin" and tries to parse them.

@ip2k

This comment has been minimized.

Copy link

ip2k commented Jul 30, 2014

[ Edited 08/17 to no longer use a temp file]

I re-wrote this to work with their current site and made the resulting daily-updated list available at http://ip2k.com/list.gz

(note that this requires hxwls which should be in the 'html-xml-utils' package or similar in your distro)


#!/bin/bash

for url in $(curl -s https://www.iblocklist.com/lists.php \
| hxwls \
| grep -v png \
| grep 'list=' \
| sed 's|/list.php?list=||g' \
| sed 's|^|http://list.iblocklist.com/?list=|g' \
| sed 's|$|\&fileformat=p2p\&archiveformat=gz|g'); do wget --no-verbose "${url}" -O - | gunzip |egrep -v '^#' >> list; done

gzip list
echo "DONE"
ls -lah list.gz

@dkavraal

This comment has been minimized.

Copy link

dkavraal commented Aug 2, 2014

This one is for uTorrent on MacOSX, if anyone needs:

#!/bin/bash
DIR="~/Library/Application Support/uTorrent"
LIST_COUNT=$(wget -q -O - http://www.iblocklist.com/lists.php | sed -n "s/.*value='\(http:.*list=.*\)'.*/\1/p" | wc -l )
CURRENT_LIST_ID=0
mv ipfilter.dat ipfilter.`date +%Y%m%d_%s`.dat 2>/dev/null
>"${DIR}/ipfilter.dat"
STARS_PRINTED=0
echo -n -e "..."
wget -q -O - http://www.iblocklist.com/lists.php | sed -n "s/.*value='\(http:.*list=.*\)'.*/\1/p" | while read LIST_URL; do
    CURRENT_LIST_ID=$[${CURRENT_LIST_ID}+1]
    STARS=$[20*${CURRENT_LIST_ID}/${LIST_COUNT}]
    STARS_LEFT=$[${STARS}-${STARS_PRINTED}]
    STARS_PRINTED=$STARS
    for ((;STARS_LEFT>0; STARS_LEFT--)); do echo -n -e "*"; done
    URL=$LIST_URL sh -c 'wget -q -O - "$URL" | gunzip -c | egrep -v "^#" | cut -d":" -f2 | egrep -v "^$" ' >> "${DIR}/ipfilter.dat" 2>/dev/null
done
echo
@enricobacis

This comment has been minimized.

Copy link

enricobacis commented Aug 4, 2014

@ip2k nice script, you just have one typo:

for url in $(cat bl); do

should be

for url in $(cat bls); do

@ip2k

This comment has been minimized.

Copy link

ip2k commented Aug 17, 2014

@enricobacis thanks for pointing that out :) I fixed the script to not use a temp file at all now, and a daily-updated list is at http://ip2k.com/list.gz

@sudhirkhanger

This comment has been minimized.

Copy link

sudhirkhanger commented Sep 9, 2014

@ip2k Just found out about the this list. Once I add the list you generated to the Transmission, it will get automatically updated as frequently are yours. Meaning I will not have to do anything except add the link to Transmission. Is that right?

@philippemorin123

This comment has been minimized.

Copy link

philippemorin123 commented Nov 21, 2014

ip2k, is your list still maintained?

@ghost

This comment has been minimized.

Copy link

ghost commented Feb 3, 2015

Hey, looks like a script pasting party!
https://github.com/cinus/iblocklist/blob/master/get_lists.sh

@LinixLinux

This comment has been minimized.

Copy link

LinixLinux commented May 24, 2015

@negativeions There are 389,000 ranges, or, as Transmission calls it, rules. 389,000 ranges = two billion individual IPs ;).

|Source:|
https://forum.transmissionbt.com/viewtopic.php?t=16987

@koesherbacon

This comment has been minimized.

Copy link

koesherbacon commented Jun 19, 2015

Do you have any idea why all torrents from LinuxTracker.org (IP: 66.135.33.31, found from linuxtracker.org.ipaddress.com) are being blocked by your list? I downloaded the list and opened it in Notepad++; however, I could not find either the URL and IP address when I searched for them, so I'm not certain why they would be blocked.

Any ideas you might have on how to fix this issue?

@ip2k

This comment has been minimized.

Copy link

ip2k commented Jul 2, 2015

http://ip2k.com/list.gz is updated daily via crontab; yes.

@Luen

This comment has been minimized.

Copy link

Luen commented Aug 3, 2015

@koesherbacon iblocklist.com is in charge of which ip's are blocked. It might be that the ip address is within an ip range that is blocked.

@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Sep 3, 2015

@ip2k I'm getting literally thousands of hits a day on this and haven't looked at it in at least a year. I'm considering just 301 redirecting to yours. Are you OK with that?

@madhouserevival

This comment has been minimized.

Copy link

madhouserevival commented Sep 8, 2015

@ip2k, so do I open it with a txt editor and copy into the ipfilter.dat file? Looks like Dave's Ipfilter hasn't been updated in almost a month. Miss the normal ipfilter updater which no longer works.

@freed00m

This comment has been minimized.

Copy link

freed00m commented Sep 17, 2015

@decabyte

This comment has been minimized.

Copy link

decabyte commented Sep 27, 2015

@freed00m is not DEAD ... it is just returning an empty list due to some URL schema changes on the original website.

To fix it just run the ip2k script changing:

| sed 's|/list.php?list=||g' \

with:

| sed 's|/list?list=||g' \

and you will get a long blocklist. :)

@petehalatsis

This comment has been minimized.

Copy link

petehalatsis commented Sep 29, 2015

So, I have some other OS that I will not name. I was just downloading the list.gz file, unzipping, and importing to my bittorrent client. I assume there is a better way to do this? Also, just downloading the list right now results in an empty file... Please assist!

@sandstrom

This comment has been minimized.

Copy link

sandstrom commented Dec 23, 2015

For anyone in search of a blocklist for Transmission etc. (ending up here via Google), here is a list: http://www.wael.name/wael.list.p2p.gz

(note that it's not related to the script in this gist)

@timfam

This comment has been minimized.

Copy link

timfam commented Jan 29, 2016

@sandstrom How often is your list updated?

@jult

This comment has been minimized.

Copy link

jult commented Feb 14, 2016

Mine's updated nightly: https://gist.github.com/jult/e76c628899bd5aa3c33a
I use https://jult.net/bloc.txt.gz in Tixati and it shows me quite a lot of matches so it's working really well!

@dementio

This comment has been minimized.

Copy link

dementio commented Mar 2, 2016

@jult I'm trying to modify yours to also use the I-Blocklist lists and add the &username=USERNAME&pin=PIN without having to put a separate line for each list but keep coming up short on how to achieve this. Any ideas?

@jult

This comment has been minimized.

Copy link

jult commented Mar 8, 2016

@dementio How many lists are there for your login? You need to auth yourself, for every list, I don't see how that would be easier in one go. Why bother with shortening that?

@Zulith

This comment has been minimized.

Copy link

Zulith commented Mar 21, 2016

Thank you jult, I've plugged your blocklist url into transmission and it has nearly 115k entries (for the tixati list), is this the one you would recommend for most users? some of your other lists have nearly 500k entries according to transmission, but I see that they have way more cruft in them. Are they both equally as effective?

@rtinikan

This comment has been minimized.

Copy link

rtinikan commented Apr 19, 2016

Hi all, @jult Thanks for the script

@p1ne

This comment has been minimized.

Copy link

p1ne commented Jan 11, 2017

Modified it a bit so it takes new format of iblocklist urls:

curl -s https://www.iblocklist.com/lists.php | sed -n "s/.*value='\(http:.*=p2p.*\)'.*/\1/p" | xargs wget -O - | gunzip | egrep -v '^#'

@glaszig

This comment has been minimized.

Copy link

glaszig commented Mar 23, 2017

created a gist from @p1ne's pipe. the result is https://silo.glasz.org/antip2p.list.gz and is updated daily. host yourself if you can as this may go away without warning.

@hobson

This comment has been minimized.

Copy link

hobson commented Oct 18, 2017

worked for me on Ubuntu (sed regex needs to be more liberal for the latest blocklist page format):

curl -s https://www.iblocklist.com/lists.php | sed -n "s/.*value='\(http[^']*\)'.*/'\1'/p" | xargs wget -O - | gunzip | egrep -v '^#' > ~/torrent_blocklist_url.txt

@JCDrew

This comment has been minimized.

Copy link

JCDrew commented Dec 14, 2017

The bitsurge.net domain has expired and the blocklist can no longer be downloaded from John Tyree's page on it.

@siemvaessen

This comment has been minimized.

Copy link

siemvaessen commented Feb 17, 2018

http://john.bitsurge.net/public/biglist.p2p.gz works just fine and updates to 411.500 ip adresses at time of writing. Unclear however how fresh that list is.

@Henrik2

This comment has been minimized.

Copy link

Henrik2 commented Mar 2, 2018

Mine says 410 998 rules and not 411 500 but it's from that same address.

@ghost

This comment has been minimized.

Copy link

ghost commented Mar 7, 2018

@siemvaessen curl -sI http://john.bitsurge.net/public/biglist.p2p.gz | grep Last-Modified gives me Last-Modified: Wed, 07 Mar 2018 08:00:30 GMT so I would guess the list is still updated?

@Henrik2

This comment has been minimized.

Copy link

Henrik2 commented Mar 12, 2018

Can't use it with qBittorent since it's a .gz file and qBittorent only supports .dat .p2p and .p2b. Anyone know what to do? Just changing the .gz file to .dat .p2p or .p2b dosent work as qBittorent just says 0 rules applied if you do that.

@Globulopolis

This comment has been minimized.

Copy link

Globulopolis commented Mar 24, 2018

@Henrik2 you can't use that list for bt client, because it's need to have a proper list format.
Below I wrote a PHP script to download hourly updated list and convert it to utorrent(qbittorent) format.

$time_start = microtime(true);

// Download and save
// See https://gist.github.com/glaszig/bf96beccf4694ae25d4f1f7cc6224985#gistcomment-2180796 for info about used URL
$arch = file_get_contents('http://o2php53.odns.fr/block.list.gz');

$fsrc = fopen('block.list.gz', 'wb');
fwrite($fsrc, $arch);
fclose($fsrc);

// Unpack
$gsfile = gzopen('block.list.gz', 'rb');
$fsrc = fopen('block.list', 'wb');

while (!gzeof($gsfile))
{
	fwrite($fsrc, gzread($gsfile, 4096));
}

fclose($fsrc);
gzclose($gsfile);
unlink('block.list.gz');

$fsrc = fopen('block.list', 'rb');
$fdst = fopen('ipfilter.dat', 'w');

$contents = fread($fsrc, filesize('block.list'));
$content = explode("\n", $contents);
$startips = array();
$endips = array();
$access = array();
$desc = array();

// Get start IPs in array format to check for duplicates
foreach ($content as $key => $row)
{
	if (!empty($row))
	{
		$lines   = explode(':', $row);
		$ipRange = explode('-', end($lines));
		$startips[$key] = $ipRange[0];
		$endips[$key]   = $ipRange[1];
		$access[$key]   = ($ipRange[0] == '127.0.0.0' || $ipRange[0] == '127.0.0.2' || $ipRange[0] == '192.0.2.0' || $ipRange[0] == '192.168.0.0') ? '0' : '000';
		$desc[$key]     = $lines[0];
	}
}

$startIPsArr = array_unique($startips);

foreach ($startIPsArr as $k => $ip)
{
	$startIPArr    = explode('.', $ip);
	$startIPArr[0] = sprintf('%03d', $startIPArr[0]);
	$startIPArr[1] = sprintf('%03d', $startIPArr[1]);
	$startIPArr[2] = sprintf('%03d', $startIPArr[2]);
	$startIPArr[3] = sprintf('%03d', $startIPArr[3]);
	$startIP = implode('.', $startIPArr);

	if (!empty($endips[$k]))
	{
		$endIPArr    = explode('.', $endips[$k]);
		$endIPArr[0] = sprintf('%03d', $endIPArr[0]);
		$endIPArr[1] = sprintf('%03d', $endIPArr[1]);
		$endIPArr[2] = sprintf('%03d', $endIPArr[2]);
		$endIPArr[3] = sprintf('%03d', $endIPArr[3]);
		$endIP = implode('.', $endIPArr);
	}

	$row = $startIP . ' - ' . $endIP . ' , ' . $access[$k] . ' , ' . $desc[$k] . "\n";
	fwrite($fdst, $row);
}

fclose($fsrc);
fclose($fdst);

echo '<a href="ipfilter.dat">Download ipfilter.dat</a>';
echo '<p><strong>Total IPs or ranges:</strong> ' . count($startIPsArr) . '</p>';

$time_end = microtime(true);
$execution_time = ($time_end - $time_start);
echo '<p><strong>Total exec time:</strong> ' . $execution_time . ' seconds</p>';

NB! Using this list in utorrent(not a qbittorrent) will slow down first run about 1 minute(because of loading 500k+ records).

UPD Remove duplicate IPs and ranges, proper access level for loopbacks.

@ninetyfivenorth

This comment has been minimized.

Copy link

ninetyfivenorth commented Apr 7, 2018

@Henrik2 just use a url shortener service or use a gist > raw saved as the file name extension desired, voila!

or try this?

sudo wget https://raw.githubusercontent.com/cinus/iblocklist/master/get_lists.sh && sh get_lists.sh

@hyptechdev2015

This comment has been minimized.

@gombosg

This comment has been minimized.

Copy link

gombosg commented Jun 29, 2018

An updated version of the gist in the beginning:
https://gist.github.com/gombosg/4577e1d14e2d52d2f16462c0ef412433

@iogitio

This comment has been minimized.

Copy link

iogitio commented Jul 4, 2018

Is it possible to get the blocklist in .p2p format hosted online somewhere? :/

@ondapc

This comment has been minimized.

Copy link

ondapc commented Aug 4, 2018

curl -s https://www.iblocklist.com/lists.php
| sed -n "s/.value='(http:.)'.*/\1/p"
| xargs wget -O -
| gunzip
| egrep -v '^#'
| gzip - > bt_blocklist.gz

@STX2k

This comment has been minimized.

Copy link

STX2k commented Nov 29, 2018

I have an subscription on iblocklist.com. Does anyone have a script also download the Premium lists and merge it into one?

@Myron-S

This comment has been minimized.

Copy link

Myron-S commented Jan 17, 2019

Looks like it's still being updated. 👍

$ curl -sI http://john.bitsurge.net/public/biglist.p2p.gz | grep Last-Modified
Last-Modified: Thu, 17 Jan 2019 08:00:21 GMT

@francisuk1989

This comment has been minimized.

Copy link

francisuk1989 commented Feb 4, 2019

The only problem i have with the level 1, level 2, level 3 from iblocklist is that is so much out of date so for example the two ranges (maybe more) 51.0.0.0/8 163.0.0.0/8 are the old gov ip range for example "51.51.107.243" is now allocated to microsoft (test if you wont to) is good in someway to block microsoft but blocking out seedboxes, vpn's and most of all genuine harmless ISPs your missing out https://www.iblocklist.com/search.php is your choice what you do but just wonted to point this out.

@mariusvw

This comment has been minimized.

Copy link

mariusvw commented Aug 10, 2019

Adjusted to work on FreeBSD.

#!/bin/sh

# Download lists, unpack and filter, write to gzipped file 
curl -s https://www.iblocklist.com/lists.php \
| sed -n "s/.*value='\(http:.*\)'.*/\1/p" \
| sed "s/\&amp;/\&/g" \
| sed "s/http/\"http/g" \
| sed "s/gz/gz\"/g" \
| xargs curl -s -L \
| gunzip \
| egrep -v '^#' \
| gzip - > /usr/local/etc/transmission/home/bt_blocklist.gz
@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Aug 12, 2019

Dear everyone, this might still be getting updated. Might not. I definitely pay zero attention to it now. Who knows what it's got in it. If you care about this kind of thing, please move on to some other project that actually maintains their lists.

@fonic

This comment has been minimized.

Copy link

fonic commented Oct 27, 2019

If you're looking for an alternative, you might want to check out my script: https://github.com/fonic/ipfilter

@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Oct 28, 2019

Nice. Yes. Use that instead, everyone.

Although, I have no idea why you'd choose to write such a complicated thing in 400 lines of bash instead of a real language.

@mariusvw

This comment has been minimized.

Copy link

mariusvw commented Oct 28, 2019

@johntyree I can understand why, if you don't have such a language available on the target machine bash can still be a good solution.

@fonic

This comment has been minimized.

Copy link

fonic commented Oct 28, 2019

Although, I have no idea why you'd choose to write such a complicated thing in 400 lines of bash instead of a real language.

There are so many reasons - because I can, because it's fun, because it's a challenge... Actually, I have developed much more complex shell scripts. For example, the init script of my initramfs project I'm about to release soon (which is ash-based due to BusyBox; an even bigger challenge) is currently at 1600 lines / 50 KiB.

In this case, however, the answer is simple: I created the ipfilter script years ago and it just became bigger over time. Normally, I'd use Python for stuff like that.

@fonic

This comment has been minimized.

Copy link

fonic commented Oct 28, 2019

I applied a couple of enhancements to make the script more user-friendly. If anyone has further suggestions, feel free to open an issue -> https://github.com/fonic/ipfilter

@Bundy01

This comment has been minimized.

Copy link

Bundy01 commented Mar 16, 2020

@johntyree
Hi,
Would it be possible to create this list in CIDR format?

This is the only format that can be compatible with my client.
If not, can you give me a link to a compatible list?

Thank you very much.

@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Mar 17, 2020

@Bundy01 I don't maintain this anymore. Try opening an issue with https://github.com/fonic/ipfilter

Good luck!

@mantrasiva

This comment has been minimized.

Copy link

mantrasiva commented Mar 31, 2020

@Bundy01 I don't maintain this anymore. Try opening an issue with https://github.com/fonic/ipfilter

Good luck!

Hi John
will be http://john.bitsurge.net/public/biglist.p2p.gz update or is gone forever ?
it was an amazing list
is there another complete list frequently update for peerblock ?
thanks

@dcalano

This comment has been minimized.

Copy link

dcalano commented Apr 12, 2020

@johntyree, Despite @fonic plugging his project here as an alternative, considering that he doesn't support Transmission (according to this issue) it is probably not a wise recommendation.

@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Apr 12, 2020

Why is supporting Transmission the most important part of this? I made a blocklist. People can use it, but I'm no longer maintaining it. That's about it.

@mantrasiva

This comment has been minimized.

Copy link

mantrasiva commented Apr 12, 2020

@johntyree
i really loved your blocklist , i used with peerblock
is there a way to update?
thanks

@fonic

This comment has been minimized.

Copy link

fonic commented Apr 12, 2020

I've been on the Internet right from the beginning, but to this day I'm still amazed by the lack of manners in some people using it. You put lots of work into something, provide it to others for free, and they're like 'oh, thanks, but let me point out what's wrong with it'.

So, @dcalano, the computer science student who should know better:
My script generates a blocklist. You want to use it for Transmission, be my guest. Transmission doesn't seem to support reading blocklists from local files like other clients do. Thus, you could head to the Transmission project, open a bug/request and ask them nicely if they would consider adding support for local blocklists. Alternatively, you could open a bug/request and tell them 'hey, you guys should stop releasing/promoting Transmission, considering that you don't support Fonic's ipfilter project' ;)

@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Apr 12, 2020

Pretty soon http://john.bitsurge.net/public/biglist.p2p.gz is likely to disappear anyway because my host is discontinuing the service plan that made hosting this file affordable. Even now, with it being way past its prime, it's still quite a lot of network egress per month.

Even if my hosting costs weren't changing, as this script gets more and more out of date it becomes less and less responsible for me to keep letting people rely on it. Maybe if their clients start failing to download they will update their configs to point to something more recent. There are lots of alternatives.

@mantrasiva

This comment has been minimized.

Copy link

mantrasiva commented Apr 12, 2020

@johntyree
thanks for the answer , so the only way to have more privacy is a vpn , seeing i can't find a good list like yours ,on the contrary i can't find a free list
thanks
@fonic
might I ask you a question script?
do I need to have several subscribtions ?
thanks

@fonic

This comment has been minimized.

Copy link

fonic commented Apr 12, 2020

@fonic
might I ask you a question script?
do I need to have several subscribtions ?
thanks

Erm, I'm not sure I understand your question. What do you mean by subscriptions?

@Koeroesi86

This comment has been minimized.

Copy link

Koeroesi86 commented Apr 12, 2020

Anyone trying to make scripts work, it would just need a User-Agent header (ex. curl -H "User-Agent: user-Agent-Name-Here"). It seems I-Blocklist trying to "protect" direct downloads.

@mantrasiva

This comment has been minimized.

Copy link

mantrasiva commented Apr 12, 2020

@fonic
sorry for my poor english
to use your scritpt do I need iblocklist subscriptions ? They are not free and do I need a subscription?
thanks

@fonic

This comment has been minimized.

Copy link

fonic commented Apr 12, 2020

@mantrasiva:
Now I got you. No, you don't need subscriptions. Most of the I-Blocklist lists are free, only some require subscriptions, but I can't tell you anything about those as I have never used them.

@Koeroesi86:
Since when? Works fine without.

@Koeroesi86

This comment has been minimized.

Copy link

Koeroesi86 commented Apr 12, 2020

@fonic
The list.php seems to need it if you want to parse from the page.

@johntyree

This comment has been minimized.

Copy link
Owner Author

johntyree commented Apr 12, 2020

@Koeroesi86 you're right, they started checking at some point. It's an easy fix and I've done so, but this is a good example of why people should be looking for alternatives. If it breaks again, people could end up downloading an empty list and not realize it.

@Naunter

This comment has been minimized.

Copy link

Naunter commented Apr 15, 2020

@mantrasiva

This comment has been minimized.

Copy link

mantrasiva commented Apr 15, 2020

@Naunter
hi
does it include p2p filters?
thank you so much!

@fonic
Hi
i don't know in your nation ,but here western europe are not free
a screenshot
https://i.imgur.com/5z68u9S.png
thanks

@Naunter

This comment has been minimized.

Copy link

Naunter commented Apr 15, 2020

@Naunter
hi
does it include p2p filters?
thank you so much!

I'm not sure what P2P filters you are looking for, but the block lists source is from https://www.iblocklist.com/lists.php. You could check it your self.

@mantrasiva

This comment has been minimized.

Copy link

mantrasiva commented Apr 15, 2020

@Naunter
HI
the link https://www.iblocklist.com/lists.php doesn't work for me ,
will you keep the the bt_blocklists.gz updated?
thanks

@Naunter

This comment has been minimized.

Copy link

Naunter commented Apr 15, 2020

@Naunter
hi
does it include p2p filters?
thank you so much!

I'm not sure what P2P filters you are looking for, but the block lists source is from https://www.iblocklist.com/lists.php. You could check it your self.
hi
the link https://www.iblocklist.com/lists.php doesn't work for me ,
will you keep the the bt_blocklists.gz updated?
thanks

You could download the .gz file to your local and unzip it to view the content in side. It is a list of rules. And the bt_blocklists.gz will be updated daily automatic as long as Github is alive.

@Cauryl

This comment has been minimized.

Copy link

Cauryl commented Apr 19, 2020

Otherwise, for those interested, my very simple script to generate the BlockList from the lists available on I-Blocklist: https://gist.github.com/Cauryl/d2414eaa6c10f4cfc946aa446f091832.

@shmup

This comment has been minimized.

Copy link

shmup commented Jun 3, 2020

i'm curious what people still using this think it's preventing? let's assume it's > 0% effective at going "unnoticed" by entity's collecting connected peer ips, but obviously fall short of 100% effectiveness. is it that you're still reducing visibility that you care about blocklists? i mean, if you really care about reduced visibility, you'll likely use a vpn, even a free one

i used a blocklist for years before i thought "..wait, what am i thinking/doing"

@YesTrustMe

This comment has been minimized.

Copy link

YesTrustMe commented Jun 16, 2020

@Koeroesi86 you're right, they started checking at some point. It's an easy fix and I've done so, but this is a good example of why people should be looking for alternatives. If it breaks again, people could end up downloading an empty list and not realize it.<

Besides a VPN and a subscription to a service like iblocklist what do you recommend? Your list has been awesome and while it's not fool proof it's often better than nothing. If anything it weeds out a lot of crap.

@YesTrustMe

This comment has been minimized.

Copy link

YesTrustMe commented Jun 16, 2020

i'm curious what people still using this think it's preventing? let's assume it's > 0% effective at going "unnoticed" by entity's collecting connected peer ips, but obviously fall short of 100% effectiveness. is it that you're still reducing visibility that you care about blocklists? i mean, if you really care about reduced visibility, you'll likely use a vpn, even a free one

i used a blocklist for years before i thought "..wait, what am i thinking/doing"

People use it to supplement their VPN. Many have received a copyright infringement email while using peer block lists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.