Create a gist now

Instantly share code, notes, and snippets.

anonymous /gist:a76df8be2c729b3afdbd Secret
Created Jul 13, 2014


I'm trying to scrape some data from one website for academical purposes. Unfortunately, they ban my IP address after 1000 requests for one day.

So, is there a hosting provider that allows changing the server's IP address often? I could do this on AWS by restarting the server after every 1000 requests – the server gets a random IP after restart. But every restart is billed as 1 hour, which would make this quite costly. Any ideas?

If anyone is concerned about the ethics of this: 1) it's for academical purposes 2) I unsuccessfully tried to contact them 3) search engines do the same thing 4) they don't have a terms of use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment