Skip to content

Instantly share code, notes, and snippets.

View arthurtalkgoal's full-sized avatar

Arthur CHAN arthurtalkgoal

View GitHub Profile
@arthurtalkgoal
arthurtalkgoal / selenium_spider.py
Created October 12, 2015 16:59 — forked from irfani/selenium_spider.py
Scrapyd with Selenium Spider
from selenium import selenium
from scrapy.spider import BaseSpider
from scrapy.http import Request
import time
import lxml.html
class SeleniumSprider(BaseSpider):
name = "selenium"
allowed_domains = ['selenium.com']
start_urls = ["http://localhost"]