Skip to content

Instantly share code, notes, and snippets.

@billy3321
Created October 24, 2015 15:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save billy3321/12af236020846df9fc05 to your computer and use it in GitHub Desktop.
Save billy3321/12af236020846df9fc05 to your computer and use it in GitHub Desktop.
2015-10-24 22:53:34+0800 [npl_ly] ERROR: Spider error processing <GET http://lis.ly.gov.tw/lylegismc/lylegismemkmout?!!FUNC400>
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/base.py", line 824, in runUntilCurrent
call.func(*call.args, **call.kw)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/task.py", line 638, in _tick
taskObj._oneWorkUnit()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/internet/task.py", line 484, in _oneWorkUnit
result = next(self._iterator)
File "/Library/Python/2.7/site-packages/scrapy/utils/defer.py", line 57, in <genexpr>
work = (callable(elem, *args, **named) for elem in iterable)
--- <exception caught here> ---
File "/Library/Python/2.7/site-packages/scrapy/utils/defer.py", line 96, in iter_errback
yield next(it)
File "/Library/Python/2.7/site-packages/scrapy/contrib/spidermiddleware/offsite.py", line 26, in process_spider_output
for x in result:
File "/Library/Python/2.7/site-packages/scrapy/contrib/spidermiddleware/referer.py", line 22, in <genexpr>
return (_set_referer(r) for r in result or ())
File "/Library/Python/2.7/site-packages/scrapy/contrib/spidermiddleware/urllength.py", line 33, in <genexpr>
return (r for r in result or () if _filter(r))
File "/Library/Python/2.7/site-packages/scrapy/contrib/spidermiddleware/depth.py", line 50, in <genexpr>
return (r for r in result or () if _filter(r))
File "/Users/billy3321/git/g0v/ly/twlyparser/data/twly/crawler/twly_crawler/spiders/npl_ly_spider.py", line 22, in parse
yield Request(urljoin(response.url, node.xpath('@href').extract_first()), callback=self.parse_ad, dont_filter=True)
exceptions.AttributeError: 'SelectorList' object has no attribute 'extract_first'
2015-10-24 22:53:34+0800 [npl_ly] INFO: Closing spider (finished)
2015-10-24 22:53:34+0800 [npl_ly] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 247,
'downloader/request_count': 1,
'downloader/request_method_count/GET': 1,
'downloader/response_bytes': 31651,
'downloader/response_count': 1,
'downloader/response_status_count/200': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2015, 10, 24, 14, 53, 34, 761086),
'log_count/DEBUG': 3,
'log_count/ERROR': 1,
'log_count/INFO': 7,
'response_received_count': 1,
'scheduler/dequeued': 1,
'scheduler/dequeued/memory': 1,
'scheduler/enqueued': 1,
'scheduler/enqueued/memory': 1,
'spider_exceptions/AttributeError': 1,
'start_time': datetime.datetime(2015, 10, 24, 14, 53, 34, 601425)}
2015-10-24 22:53:34+0800 [npl_ly] INFO: Spider closed (finished)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment