This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
# author : Conred Wang | |
# title : Wed scraping using Scrapy. Code 2 of 3. skyscraper_spider.py | |
from scrapy import Spider | |
from scrapy.selector import Selector | |
from skyscraper.items import SkyscraperItem | |
import scrapy |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding: utf-8 -*- | |
# author : Conred Wang | |
# title : Wed scraping using Scrapy. Code 2 of 3. pipelines.py | |
class WriteItemPipeline(object): | |
def __init__(self): | |
self.filename = 'tall.txt' | |
def open_spider(self, spider): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-*- coding: utf-8 -*- | |
# author : Conred Wang | |
# title : Wed scraping using Scrapy. Code 1 of 3. items.py | |
from scrapy import Item, Field | |
# SkyscraperItem has information about one skyscraper/building: | |
# Type-1. Some fields are from primary page "https://skyscrapercenter.com/buildings". | |
# Type-2. Some fields are from secondary pages by navigating the skyscraper's/building's specific url. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
--- | |
title: “I Told You So!" | |
author: “Conred Wang“ | |
date: “10/10/2016” | |
output: | |
ioslides_presentation: | |
logo: assets/img/i.told.you.so.jpg | |
css: assets/css/ioslides.css | |
abstract: | |
* R visualization using gglot2. |