Skip to content

Instantly share code, notes, and snippets.

@ShaikeA
Last active January 13, 2019 19:59
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ShaikeA/b63faab3b131a60e03d2c1d19ae1c306 to your computer and use it in GitHub Desktop.
Save ShaikeA/b63faab3b131a60e03d2c1d19ae1c306 to your computer and use it in GitHub Desktop.
def proxies_pool():
url = 'https://www.sslproxies.org/'
# Retrieve the site's page. The 'with'(Python closure) is used here in order to automatically close the session when done
with requests.Session() as res:
proxies_page = res.get(url)
# Create a BeutifulSoup object and find the table element which consists of all proxies
soup = BeautifulSoup(proxies_page.content, 'html.parser')
proxies_table = soup.find(id='proxylisttable')
# Go through all rows in the proxies table and store them in the right format (IP:port) in our proxies list
proxies = []
for row in proxies_table.tbody.find_all('tr'):
proxies.append('{}:{}'.format(row.find_all('td')[0].string, row.find_all('td')[1].string))
return proxies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment