Skip to content

Instantly share code, notes, and snippets.

@fyoorer
Last active October 20, 2015 14:44
Show Gist options
  • Save fyoorer/a72d4ec73911b5859098 to your computer and use it in GitHub Desktop.
Save fyoorer/a72d4ec73911b5859098 to your computer and use it in GitHub Desktop.
Fetch links from bugcrowd's list
import requests
from bs4 import BeautifulSoup
bugcrowdlist=requests.get('https://bugcrowd.com/list-of-bug-bounty-programs/')
soup=BeautifulSoup(bugcrowdlist.content)
#print soup
anchors=soup.findAll("a", { "class" : "tracked" })
fil=open('alldata.txt','w')
print "Please wait while I fetch the data..."
data=dict()
for tag in anchors:
link = tag.get('href',None)
if link.startswith('http'):
fil.writelines(link+'\n')
fil.close()
print data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment