URL Permutation Generator

readme.md

About

This little script helps generate URL permutations. A simple example is opening a few pages that have some numerical range part across servers that also have a numerical range part.

Ranges are defined by square brackets ([ & ]) and are passed directly to Python's range function. To generate the range 0 you would use [1] and to generate the range 1,2,3,4,5 you would use [1,6]. You may also pass a step just like you would when calling range directly: [1,11,2] = 1,3,5,7,9.

Example Usage

python url_perm.py http://server[1,7].example.com/page/[20,22]/
http://server1.example.com/page/20/
http://server1.example.com/page/21/
http://server2.example.com/page/20/
http://server2.example.com/page/21/
http://server3.example.com/page/20/
http://server3.example.com/page/21/
http://server4.example.com/page/20/
http://server4.example.com/page/21/
http://server5.example.com/page/20/
http://server5.example.com/page/21/
http://server6.example.com/page/20/
http://server6.example.com/page/21/

url_perm.py

from itertools import product
import re
import sys

# match ranges in brackets as single digits or pairs separated by commas
num_patt = re.compile('\[([\d,]+)\]')

def parse_url(url):
    """
    Parses range strings from the given URL and returns a list of all URL
    permutations.
    
    >>> url = 'http://server[1,4].example.com'
    >>> urls = parse_url(url)
    >>> for url in urls:
    ...     print url
    http://server1.example.com
    http://server2.example.com
    http://server3.example.com
    
    >>> url = 'http://server[1,4].example.com/page/[10,12]/'
    >>> urls = parse_url(url)
    >>> for url in urls:
    ...     print url
    http://server1.example.com/page/10/
    http://server1.example.com/page/11/
    http://server2.example.com/page/10/
    http://server2.example.com/page/11/
    http://server3.example.com/page/10/
    http://server3.example.com/page/11/
    """
    urls = []
    ranges = []
    for range_str in num_patt.findall(url):
        # replace this current range string with a digit format placeholder
        url = url.replace('[%s]' % range_str, '%d', 1)
        # make a tuple of the found range
        range_tpl = tuple([int(x) for x in range_str.split(',')])
        # push it in our ranges list
        ranges.append(range_tpl)
    
    # ranges is just a list of tuples at this point so we need to make them ranges
    ranges = [xrange(*r) for r in ranges]
    
    # create the url permutations
    perms = [p for p in product(*tuple(ranges))]
    for perm in perms:
        urls.append(url % perm)
    
    return urls
    
    

if __name__ == '__main__':
    if len(sys.argv) == 2:
        url = sys.argv[1]
    else:
        print 'Expected 1 argument, a URL'
        sys.exit(1)
    urls = parse_url(url)
    for url in urls:
        print url
    sys.exit(0)