Skip to content

Instantly share code, notes, and snippets.

/gist:a76df8be2c729b3afdbd Secret

Created July 13, 2014 12:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save anonymous/a76df8be2c729b3afdbd to your computer and use it in GitHub Desktop.
Save anonymous/a76df8be2c729b3afdbd to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.md

I'm trying to scrape some data from one website for academical purposes. Unfortunately, they ban my IP address after 1000 requests for one day.

So, is there a hosting provider that allows changing the server's IP address often? I could do this on AWS by restarting the server after every 1000 requests – the server gets a random IP after restart. But every restart is billed as 1 hour, which would make this quite costly. Any ideas?

If anyone is concerned about the ethics of this: 1) it's for academical purposes 2) I unsuccessfully tried to contact them 3) search engines do the same thing 4) they don't have a terms of use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment