Skip to content

Instantly share code, notes, and snippets.

Alireza Mika alirezamika

Block or report user

Report or block alirezamika

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View auto-scraper-api.md

Hi guys. I made this API for automatic web scraping to make scraping easy. It gets a url or html content of a web page and a list of sample data which we want to scrape from that page. then it learns the scraping rules and returns the similar elements and an id for future requests.

example: say we want to fetch all related post titles in a stackoverflow page. We'll send a POST request as below:

curl -X POST http://autoscraper.ml/api -d '{"url": "https://stackoverflow.com/questions/2081586/web-scraping-with-python", "wanted_list": ["Calling an external command in Python", "What are metaclasses in Python?"]}'

It will return:

View bipedal-es-gist
Algorithm: Evolution Strategies
Code & explanation available at:
https://github.com/alirezamika/bipedal-es
You can’t perform that action at this time.