Skip to content

Instantly share code, notes, and snippets.

@kohn kohn/example.py Secret
Created May 12, 2016

Embed
What would you like to do?
def parse(self, response):
if banned():
yield Request(url=response.url,
meta={"change_proxy": True},
callback=self.parse)
else:
pass
@ickma

This comment has been minimized.

Copy link

ickma commented Jun 6, 2016

你好,banned并不是scrapy的内置方法,怎么识别被ban?无意中发现的这个gist,正是我今天在思考的问题,github已经follow了你!呵呵

@xujipm

This comment has been minimized.

Copy link

xujipm commented Jan 5, 2017

band() 是你自己根据趴下来的内容进行判断是否被ban!

@mugbya

This comment has been minimized.

Copy link

mugbya commented Nov 8, 2017

哦,我这被 ban 了连网站都打不开,都没内容...

@youyiqin

This comment has been minimized.

Copy link

youyiqin commented May 2, 2018

您好!meta={"change_proxy": True},元数据设置这个值,会引发什么才能更换代理?我找到了免费代理池,再由flask提供API,写入代理中间件,每次请求,都会调用一次中间件是吗?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.