Skip to content

Instantly share code, notes, and snippets.

@MinweiShen
Created February 26, 2016 18:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save MinweiShen/e085db8a44ce7431fac1 to your computer and use it in GitHub Desktop.
Save MinweiShen/e085db8a44ce7431fac1 to your computer and use it in GitHub Desktop.
scrapy json 保存中文
import json,codecs
class WebCrawlerScrapyPipeline(object):
def __init__(self):
self.file = codecs.open('item.json', 'wb', encoding='utf-8')
def process_item(self, item, spider):
line = json.dumps(dict(item)) + '\n'
# print line
self.file.write(line.decode("unicode_escape"))
return item
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment