cockcrow
cockcrow
/ dianping_ua_test.html
Created
June 4, 2016 16:42
curl 'http://www.dianping.com/search/keyword/279/10_%E8%B1%A1%E5%B1%B1%E8%85%8A%E6%8E%92%E9%AA%A8/p2' -H 'User-Agent: Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11'
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!DOCTYPE html> | |
| <html> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2016-06-10 14:35:15 [twisted] CRITICAL: Unhandled Error | |
| Traceback (most recent call last): | |
| File "/home/aesthetic/crawler_venv/local/lib/python2.7/site-packages/scrapy/commands/crawl.py", line 58, in run | |
| self.crawler_process.start() | |
| File "/home/aesthetic/crawler_venv/local/lib/python2.7/site-packages/scrapy/crawler.py", line 280, in start | |
| reactor.run(installSignalHandlers=False) # blocking call | |
| File "/home/aesthetic/crawler_venv/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1194, in run | |
| self.mainLoop() | |
| File "/home/aesthetic/crawler_venv/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1203, in mainLoop | |
| self.runUntilCurrent() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # -*- coding: utf-8 -*- | |
| from __future__ import absolute_import, print_function, unicode_literals | |
| import lxml.html | |
| from lxml.html.clean import Cleaner | |
| try: | |
| import cchardet as chardet | |
| except ImportError: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # -*- coding: utf-8 -*- | |
| from __future__ import absolute_import, print_function, unicode_literals | |
| from threading import RLock | |
| PREV, NEXT, KEY, VALUE = 0, 1, 2, 3 # names for the item fields | |
| __all__ = ['LRUCache'] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| INFO global: Vagrant version: 2.2.0 | |
| INFO global: Ruby version: 2.4.4 | |
| INFO global: RubyGems version: 2.6.14.1 | |
| INFO global: VAGRANT_INSTALLER_VERSION="2" | |
| INFO global: VAGRANT_INSTALLER_ENV="1" | |
| INFO global: VAGRANT_EXECUTABLE="/opt/vagrant/embedded/gems/2.2.0/gems/vagrant-2.2.0/bin/vagrant" | |
| INFO global: VAGRANT_INSTALLER_EMBEDDED_DIR="/opt/vagrant/embedded" | |
| INFO global: VAGRANT_LOG="debug" | |
| WARN global: resolv replacement has not been enabled! | |
| DEBUG global: Loading core plugin: /opt/vagrant/embedded/gems/2.2.0/gems/vagrant-2.2.0/plugins/synced_folders/nfs/plugin.rb |