gistfile1.txt

➜  web2mob.scraper  bin/scraper
2013-01-21 10:27:05+0100 [scrapy] INFO: Found page: TMP Norge
2013-01-21 10:27:06+0100 [scrapy] INFO: Found page: Lerøy | TMP Norge
2013-01-21 10:27:06+0100 [scrapy] INFO: Found page: Julen 2012 | TMP Norge
2013-01-21 10:27:06+0100 [scrapy] INFO: Found page: Stavanger Plastikkirurgi | TMP Norge
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: NELFO og Norsk Teknologi | TMP Norge
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: DigiPublishing | Facebookspesialisten
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: TMP Blogg | TMP Norge
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: Hektisk høst | TMP Norge
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: Poggenpohl Studio Oslo | TMP Norge
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: Seeyoublog.no | TMP Norge
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: Grilstad | Meny | TMP Norge
2013-01-21 10:27:07+0100 [scrapy] INFO: Found page: Elverum sikkerhet & service | TMP Norge
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Dekk & Felg Trondheim | TMP Norge
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Enkel redigering | DigiPublishing
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Om oss | DigiPublishing
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Skreddersydd | DigiPublishing
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Skjemabygger | DigiPublishing
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Kontakt oss | DigiPublishing
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Fotograf Eidsmo | TMP Norge
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: DigiPage | DigiPublishing
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Facebook Tmpnorge Godt nyttår | TMP Norge
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Toyota Hell Bil Sommerkampanje | TMP Norge
2013-01-21 10:27:08+0100 [scrapy] INFO: Found page: Blogg | TMP Norge
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: Facebook | DigiPublishing
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: Referanser | TMP Norge
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: Kampanje | TMP Norge
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: Webprosjekter | TMP Norge
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: 2012  januar | TMP Norge
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: Ta kontakt | TMP Norge
2013-01-21 10:27:09+0100 [scrapy] INFO: Found url: http://digipub.no/feed/
2013-01-21 10:27:09+0100 [followall] INFO: Closing spider (finished)
2013-01-21 10:27:09+0100 [followall] INFO: Dumping Scrapy stats:
    {'downloader/request_bytes': 2979,
	 'downloader/request_count': 12,
	 'downloader/request_method_count/GET': 12,
	 'downloader/response_bytes': 44329,
	 'downloader/response_count': 12,
	 'downloader/response_status_count/200': 9,
	 'downloader/response_status_count/301': 3,
	 'finish_reason': 'finished',
	 'finish_time': datetime.datetime(2013, 1, 21, 9, 27, 9, 749145),
	 'item_scraped_count': 9,
	 'request_depth_max': 2,
	 'response_received_count': 9,
	 'scheduler/dequeued': 12,
	 'scheduler/dequeued/memory': 12,
	 'scheduler/enqueued': 12,
	 'scheduler/enqueued/memory': 12,
	 'start_time': datetime.datetime(2013, 1, 21, 9, 27, 4, 468733)}
2013-01-21 10:27:09+0100 [followall] INFO: Spider closed (finished)
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: Nettside | TMP Norge
2013-01-21 10:27:09+0100 [scrapy] INFO: Found page: 5 tips for flere «likes»  | TMP Norge
2013-01-21 10:27:10+0100 [scrapy] INFO: Found page: Zoopartner | TMP Norge
2013-01-21 10:27:10+0100 [scrapy] INFO: Found page: Blogg | TMP Norge
2013-01-21 10:27:10+0100 [scrapy] INFO: Found page: Passerer 4 milliarder | TMP Norge
2013-01-21 10:27:10+0100 [scrapy] INFO: Found page: Logo | TMP Norge
2013-01-21 10:27:10+0100 [scrapy] INFO: Found page: Profil | TMP Norge
2013-01-21 10:27:10+0100 [scrapy] INFO: Found page: Identitet | TMP Norge
2013-01-21 10:27:10+0100 [scrapy] INFO: Found page: Branding | TMP Norge
2013-01-21 10:27:11+0100 [scrapy] INFO: Found page: Hva er Google+ | TMP Norge
2013-01-21 10:27:11+0100 [scrapy] INFO: Found page: Modern Design | TMP Norge
2013-01-21 10:27:11+0100 [scrapy] INFO: Found page: Myrens Sportssenter | TMP Norge
2013-01-21 10:27:11+0100 [scrapy] INFO: Found page: Rosenborg Ballklubb | TMP Norge
2013-01-21 10:27:11+0100 [scrapy] INFO: Found page: Midtnorsk Lift | TMP Norge
2013-01-21 10:27:11+0100 [scrapy] INFO: Found page: Modena Fliser Trondheim | TMP Norge
2013-01-21 10:27:12+0100 [scrapy] INFO: Found page: Fornebuklinikken | TMP Norge
2013-01-21 10:27:12+0100 [scrapy] INFO: Found page: Psykolog Anna Skjelbred | TMP Norge
2013-01-21 10:27:12+0100 [scrapy] INFO: Found page: Rea Media blir TmP Norge | TMP Norge
2013-01-21 10:27:12+0100 [scrapy] INFO: Found page: Nettbutikk | TMP Norge
2013-01-21 10:27:12+0100 [scrapy] INFO: Found page: Design | TMP Norge
2013-01-21 10:27:12+0100 [scrapy] INFO: Found page: Kundeuttalelser | TMP Norge
2013-01-21 10:27:13+0100 [scrapy] INFO: Found page: Innblikk i Facebook Timeline | TMP Norge
2013-01-21 10:27:13+0100 [scrapy] INFO: Found page: 2012  mars | TMP Norge
2013-01-21 10:27:13+0100 [scrapy] INFO: Found page: 2012  august | TMP Norge
2013-01-21 10:27:13+0100 [scrapy] INFO: Found page: 2012  september | TMP Norge
2013-01-21 10:27:13+0100 [scrapy] INFO: Found page: 2012  juni | TMP Norge
2013-01-21 10:27:13+0100 [scrapy] INFO: Found page: 2012  november | TMP Norge
2013-01-21 10:27:13+0100 [scrapy] INFO: Found page: 2013  januar | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Instagram | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Tmp Sommerlukket | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Analytics / ROI | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Facebook | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Webdesign | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Facebook  | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Google  | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Ansatte | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Medierådgivning | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Vår visjon og våre verdier | TMP Norge
2013-01-21 10:27:14+0100 [scrapy] INFO: Found page: Om TmP | TMP Norge
2013-01-21 10:27:15+0100 [followall] INFO: Closing spider (finished)
2013-01-21 10:27:15+0100 [followall] INFO: Dumping Scrapy stats:
	{'downloader/request_bytes': 17760,
	 'downloader/request_count': 64,
	 'downloader/request_method_count/GET': 64,
	 'downloader/response_bytes': 320263,
	 'downloader/response_count': 64,
	 'downloader/response_status_count/200': 60,
	 'downloader/response_status_count/301': 1,
	 'downloader/response_status_count/404': 3,
	 'finish_reason': 'finished',
	 'finish_time': datetime.datetime(2013, 1, 21, 9, 27, 15, 3079),
	 'item_scraped_count': 60,
	 'request_depth_max': 4,
	 'response_received_count': 63,
	 'scheduler/dequeued': 64,
	 'scheduler/dequeued/memory': 64,
	 'scheduler/enqueued': 64,
	 'scheduler/enqueued/memory': 64,
	 'start_time': datetime.datetime(2013, 1, 21, 9, 27, 4, 469677)}
2013-01-21 10:27:15+0100 [followall] INFO: Spider closed (finished)