Skip to content

Instantly share code, notes, and snippets.

@dmiedema
Created March 23, 2013 21:06
Show Gist options
  • Save dmiedema/5229353 to your computer and use it in GitHub Desktop.
Save dmiedema/5229353 to your computer and use it in GitHub Desktop.
A python script to parse the IP assignment/allotment list from Arin ftp://ftp.arin.net/pub/stats/arin/delegated-arin-latest as well as a bash wrapper designed to wrap the python script enabling different options. It creates a lot of unnecessary lines that could be combined when determining the subnet type. I haven't fixed that yet so in its curr…
#!/bin/bash
# Set up some default values in case options aren't specified.
DEFAULTDOWNLOADFILE="arin-current-ip-download.txt"
DEFAULTOUTPUTFILE="formatted-ips.txt"
DEFAULTDOWNLOADURL="ftp://ftp.arin.net/pub/stats/arin/delegated-arin-latest"
DEFAULTOUTPUTFILETYPE="txt" # this doesn't do anything yet, but this flag could determine other scripts to be called if warranted.
VERBOSE=0
cmd(){ echo `basename $0`; }
# usage. Self explanitory I feel. Just shows you how to use the script.
usage(){
echo "\
`cmd` [OPTION...]
-u; specify download URL. (default: $DEFAULTDOWNLOADURL)
-d; specify filename to download to. (default: $DEFAULTDOWNLOADFILE)
-s; specfiy filename to save formatted output to. (default: $DEFAULTOUTPUTFILE)
-f; alias for -s
-t; specifies the type of output you want the formatted text to be such as apache for an htaccess config file
-v; Enable verbose output.
" | column -t -s ";"
}
error(){
echo "`cmd`: invalid option -- '$1'";
echo "Try '`cmd` -h' for more information.";
exit 1;
}
# arg checks. Arguments are designed to be used as
# -flag [argument] -flag [argument].
# Thats how I always call scripts. I don't know if -dftv [arg] [arg] [arg] will work.
# haven't tested. Probably should check that at some point. But until that day comes
# this note shall remain. it might blow up.
while getopts ":u:d:s:f:t::vh" opt; do
case $opt in
h)
echo "`usage`"
exit 1
;;
u)
URL=$OPTARG
;;
d)
DOWNLOADFILE=$OPTARG
;;
s)
OUTPUTFILE=$OPTARG
;;
f)
OUTPUTFILE=$OPTARG
;;
t)
OUTPUTTYPE=$OPTARG
;;
v)
VERBOSE=1
;;
\?)
echo "Invalid Option: -$OPTARG" >&2
exit 1
;;
:)
echo "Option -$OPTARG requires an arugment." >&2
exit 1
;;
esac
done
# Assigning values. Just check to see if these variables are empty. Should make sense, yes?
if [ -z $URL ];
then
URL=$DEFAULTDOWNLOADURL;
fi
if [ -z $DOWNLOADFILE ];
then
DOWNLOADFILE=$DEFAULTDOWNLOADFILE;
fi
if [ -z $OUTPUTFILE ];
then
OUTPUTFILE=$DEFAULTOUTPUTFILE;
fi
if [ -z $OUTPUTTYPE ];
then
OUTPUTTYPE=$DEFAULTOUTPUTFILETYPE;
fi
if [ $VERBOSE -eq 1 ] #verbose output kind of sucks right now.
then
echo "URL: $URL";
echo "Download to File: $DOWNLOADFILE";
echo "Outout File: $OUTPUTFILE";
echo "Output File Type: $OUTPUTTYPE";
fi
curl $URL | cat > $DOWNLOADFILE #curl it to a file. This is just how I download things. There might/probably is a better way.
# This is where a type check could go to determine if a different script needed to be called.
python parseArin.py $DOWNLOADFILE $OUTPUTFILE
#!/bin/python
import sys #so i can use sys.argv[i]
# open up arg 1
with open(sys.argv[1], 'r') as f:
content = f.readlines() # get it, get it all.
f.close()
def conv(x):
for i in range(32):
if(2**i == int(x)):
return (32 - i) # if the number fits nicely, return it
return ((x / 256) * - 1) # if not, return a negative number
def addNet(ip, nettype):
if (nettype == 'B'):
c = 1
else:
c = 2
iparr = ip.split('.')
if(len(iparr) < 4):
return "error with ip: " + ip # Self explanatory, i hate throwing exceptions though so i just return bad info. Probably not the best idea.
n = int(iparr[c])
if (n == 255):
iparr[c] = 0
if(int(iparr[c-1]) == 255 and c == 2):
if(int(iparr[c-2]) == 255):
return "Error IP out of range: " + ip
iparr[c-2] = int(iparr[c-2]) + 1
iparr[c-1] = 0
else:
iparr[c-1] = int(iparr[c-1]) + 1
else:
iparr[c] = n + 1
return str(iparr[0]) + '.' + str(iparr[1]) + '.' + str(iparr[2]) + '.0'
def getFullString(ip, validIps):
if(validIps % 256 > 0): # the number should always be mod-able by 256, or at least when I tested it, it always was
return "error, invalid IP: " + validIps # again, return bad info when something is wrong.
nets = conv(validIps)
if(nets > 0): # it fit nicely. Awesome. Print it.
return str(ip) + '/' + str(nets)
# Boo, it didn't fit nicely.
nets = nets * -1
c = nets % 256
b = (nets - c) / 256
ret=''
for i in range(b):
ret = ret + str(ip) + '/16\n'
ip = addNet(ip, 'B')
for i in range(c):
ret = ret + str(ip) + '/24\n'
ip = addNet(ip, 'C')
return ret
output = open(sys.argv[2], 'w+')
for string in content:
spl = string.split('|')
# Change these as needed, I only wanted a list of US IPs that fit IPV4
# So it could potentially be uses as an apache config to only allow
# US based IPs in. Line format I'm looking at/care about from
# the Arin downlaoded file is as follows.
# arin|US|ipv4|65.24.0.0|262144|20000822|allocated
# from ftp://ftp.arin.net/pub/stats/arin/delegated-arin-latest
if(spl[1].upper() == 'US' and spl[2].upper() == 'IPV4'):
output.write(getFullString(spl[3], int(spl[4])) + '\n')
#One note, this does print a few extra blank lines. Haven't fixed that. Meh.
output.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment