Skip to content
Create a gist now

Instantly share code, notes, and snippets.

anonymous /gist:a76df8be2c729b3afdbd secret

I'm trying to scrape some data from one website for academical purposes. Unfortunately, they ban my IP address after 1000 requests for one day.

So, is there a hosting provider that allows changing the server's IP address often? I could do this on AWS by restarting the server after every 1000 requests – the server gets a random IP after restart. But every restart is billed as 1 hour, which would make this quite costly. Any ideas?

If anyone is concerned about the ethics of this: 1) it's for academical purposes 2) I unsuccessfully tried to contact them 3) search engines do the same thing 4) they don't have a terms of use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.