Skip to content

Instantly share code, notes, and snippets.

@mspalex
Created May 4, 2016 18:14
Show Gist options
  • Save mspalex/50bfe3c35519be253737bf6a58aec28e to your computer and use it in GitHub Desktop.
Save mspalex/50bfe3c35519be253737bf6a58aec28e to your computer and use it in GitHub Desktop.
Script to verify if .htaccess of a website is blocking Libwww-Perl and the code used to block it from inside .htaccess file.
#!/usr/bin/perl
#******************************************************
#
# THIS SCRIPT IS USED TO VERIFY .HTACCESS LIBWWW-PERL BLOCKING
# you need to add the code to .htaccess to block perl libwww
#
# 1. add the following code to .htaccess
# inside <IfModule mod_rewrite.c> after RewriteEngine On
# like the example bellow
#
# <IfModule mod_rewrite.c>
# RewriteEngine On
# ...
# RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
# RewriteRule .* – [F,L]
# ...
# </IfModule>
#
# 2. chmod +x libwww-perl-verify.pl
#
# 3. ./libwww-perl-verify.pl
# if responds 403 forbidden the blocking is OK
# if other stuff appears, blocking is NOT OK
#
#******************************************************
# Simple LWP browser for testing
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$ua->agent("$0/0.1 " . $ua->agent);
# $ua->agent("Mozilla/8.0") # pretend we are very capable browser
$req = HTTP::Request->new(GET => 'http://www.website.com/');
$req->header('Accept' => 'text/html');
# send request
$res = $ua->request($req);
# check the outcome
if ($res->is_success) {
print $res->content;
} else {
print "Error: " . $res->status_line . "\n";
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment