Skip to content

Instantly share code, notes, and snippets.

@wtrevino
wtrevino / proxymesh_middleware.py
Last active Jan 31, 2016
How to set up Scrapy middleware with Proxymesh (or any other proxy service really...)
View proxymesh_middleware.py
class ProxyMeshMiddleware(object):
def process_request(self, request, spider):
endpoint = os.environ.get('PROXY_ENDPOINT')
user = os.environ.get('PROXY_USER')
pw = os.environ.get('PROXY_PW')
if 'proxy' in request.meta:
return