Skip to content

Instantly share code, notes, and snippets.

Avatar
💭
I may be slow to respond.

Sheryar Khan thesheryar

💭
I may be slow to respond.
View GitHub Profile
View Python Extractor (Slow)
# Get all the URLs from Sitemap
from bs4 import BeautifulSoup
from requests import get
import re
import csv
def ScrapPage(urlMain):
response = get(urlMain,headers=headers)
content = BeautifulSoup(response.content, "lxml")
urls = content.find_all("loc")