Skip to content

Instantly share code, notes, and snippets.

@tecknoh19

tecknoh19/proxy_scrape.py

Created December 17, 2018 00:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tecknoh19/299e4fafaa776f565fc53fa31a249635 to your computer and use it in GitHub Desktop.
Save tecknoh19/299e4fafaa776f565fc53fa31a249635 to your computer and use it in GitHub Desktop.
Download ZIP
Web scraper to get all SOCKS5 proxies from hidemyna.me with ping under 1000 MS
Raw
proxy_scrape.py
from bs4 import BeautifulSoup
import re
import cfscrape
scraper = cfscrape.create_scraper() # returns a CloudflareScraper instance
start = 0
increment = 64
max = 256
for counter in range(0,265,64):
content = scraper.get("https://hidemyna.me/en/proxy-list/?type=5&start=" + str(counter) + "#list").content # => "<!DOCTYPE html><html><head>..."
soup = BeautifulSoup(content,"html5lib");
tbody = soup.find("tbody")
tr = tbody.findAll("tr")
for t in tr:
td = t.findAll("td")
ip = td[0].text
port = td[1].text
ping = re.sub("[^0-9]", "", td[3].text)
ping = int(ping)
if ping < 1000:
print ip + " : " + port
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment