Skip to content

Instantly share code, notes, and snippets.

@sharan-naribole
Created January 5, 2017 21:30
Show Gist options
  • Save sharan-naribole/d1c6e7a1e68931dc764c28963aeb6f07 to your computer and use it in GitHub Desktop.
Save sharan-naribole/d1c6e7a1e68931dc764c28963aeb6f07 to your computer and use it in GitHub Desktop.
def parse_submission(self, response):
self.logger.info("Visited %s", response.url)
item = response.meta['item']
item['title'] = response.css('.title::text').extract()[1]
item['score'] = response.css('.likes::text').extract_first()
item['flair_map'] = {}
for tagline in response.css('.tagline')[1:]: #first is submission poster
redditor = tagline.css('a::text').extract()
if len(redditor)==3:
redditor = redditor[1]
if(redditor not in list(item['flair_map'].keys())):
flair = tagline.css('.flair::text').extract_first()
if flair:
item['flair_map'][redditor] = flair
yield(item)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment