gistfile1.txt

2017-03-23 20:25:00 [scrapy.extensions.logstats] INFO: Crawled 2631 pages (at 556 pages/min), scraped 96 items (at 25 items/min)
2017-03-23 20:25:06 [anansi.dao.frontier] INFO: ['infocenter.arm.com', 'landmark.cs.cornell.edu', 'dblp.uni-trier.de', 'aclanthology.info', 'events.cornell.edu']
2017-03-23 20:25:06 [anansi.dao.frontier] INFO: Dequeuing batch of frontier URIs; frontier size 912644; select using sample {TABLESAMPLE BERNOULLI(0.10957174977318648)}, dominant {AND uri NOT LIKE '%infocenter.arm.com%' AND uri NOT LIKE '%landmark.cs.cornell.edu%' AND uri NOT LIKE '%dblp.uni-trier.de%' AND uri NOT LIKE '%aclanthology.info%' AND uri NOT LIKE '%events.cornell.edu%'}
2017-03-23 20:25:07 [anansi.dao.frontier] INFO: Populated cache with 737 frontier URIs
2017-03-23 20:26:00 [scrapy.extensions.logstats] INFO: Crawled 3055 pages (at 424 pages/min), scraped 113 items (at 17 items/min)
2017-03-23 20:26:28 [anansi.dao.frontier] INFO: ['infocenter.arm.com', 'aclanthology.info', 'events.cornell.edu', 'dblp.uni-trier.de', 'digitalcommons.unl.edu', 'www.aaai.org']
2017-03-23 20:26:28 [anansi.dao.frontier] INFO: Dequeuing batch of frontier URIs; frontier size 911131; select using sample {TABLESAMPLE BERNOULLI(0.1097537017179747)}, dominant {AND uri NOT LIKE '%infocenter.arm.com%' AND uri NOT LIKE '%aclanthology.info%' AND uri NOT LIKE '%events.cornell.edu%' AND uri NOT LIKE '%dblp.uni-trier.de%' AND uri NOT LIKE '%digitalcommons.unl.edu%' AND uri NOT LIKE '%www.aaai.org%'}
2017-03-23 20:26:29 [anansi.dao.frontier] INFO: Populated cache with 653 frontier URIs
2017-03-23 20:26:59 [scrapy.extensions.logstats] INFO: Crawled 3621 pages (at 566 pages/min), scraped 145 items (at 32 items/min)
2017-03-23 20:27:29 [anansi.dao.frontier] INFO: ['www.aaai.org', 'digitalcommons.unl.edu', 'infocenter.arm.com', 'landmark.cs.cornell.edu', 'events.cornell.edu']
2017-03-23 20:27:29 [anansi.dao.frontier] INFO: Dequeuing batch of frontier URIs; frontier size 910100; select using sample {TABLESAMPLE BERNOULLI(0.1098780353807274)}, dominant {AND uri NOT LIKE '%www.aaai.org%' AND uri NOT LIKE '%digitalcommons.unl.edu%' AND uri NOT LIKE '%infocenter.arm.com%' AND uri NOT LIKE '%landmark.cs.cornell.edu%' AND uri NOT LIKE '%events.cornell.edu%'}
2017-03-23 20:27:30 [anansi.dao.frontier] INFO: Populated cache with 684 frontier URIs
2017-03-23 20:27:59 [scrapy.extensions.logstats] INFO: Crawled 4110 pages (at 489 pages/min), scraped 171 items (at 26 items/min)
2017-03-23 20:28:37 [anansi.dao.frontier] INFO: ['infocenter.arm.com', 'aclanthology.info', 'www.aaai.org', 'digitalcommons.unl.edu', 'dblp.uni-trier.de', 'landmark.cs.cornell.edu']
2017-03-23 20:28:37 [anansi.dao.frontier] INFO: Dequeuing batch of frontier URIs; frontier size 909361; select using sample {TABLESAMPLE BERNOULLI(0.10996732870664125)}, dominant {AND uri NOT LIKE '%infocenter.arm.com%' AND uri NOT LIKE '%aclanthology.info%' AND uri NOT LIKE '%www.aaai.org%' AND uri NOT LIKE '%digitalcommons.unl.edu%' AND uri NOT LIKE '%dblp.uni-trier.de%' AND uri NOT LIKE '%landmark.cs.cornell.edu%'}
2017-03-23 20:28:37 [anansi.dao.frontier] INFO: Populated cache with 647 frontier URIs
2017-03-23 20:28:59 [scrapy.extensions.logstats] INFO: Crawled 4663 pages (at 553 pages/min), scraped 208 items (at 37 items/min)
2017-03-23 20:29:34 [anansi.dao.frontier] INFO: ['infocenter.arm.com', 'digitalcommons.unl.edu', 'dblp.uni-trier.de', 'events.cornell.edu']
2017-03-23 20:29:34 [anansi.dao.frontier] INFO: Dequeuing batch of frontier URIs; frontier size 907389; select using sample {TABLESAMPLE BERNOULLI(0.11020631724651721)}, dominant {AND uri NOT LIKE '%infocenter.arm.com%' AND uri NOT LIKE '%digitalcommons.unl.edu%' AND uri NOT LIKE '%dblp.uni-trier.de%' AND uri NOT LIKE '%events.cornell.edu%'}
2017-03-23 20:29:34 [anansi.dao.frontier] INFO: Populated cache with 705 frontier URIs
2017-03-23 20:29:59 [scrapy.extensions.logstats] INFO: Crawled 5148 pages (at 485 pages/min), scraped 231 items (at 23 items/min)
2017-03-23 20:30:50 [anansi.dao.frontier] INFO: ['infocenter.arm.com', 'aclanthology.info', 'events.cornell.edu', 'www.aaai.org', 'digitalcommons.unl.edu', 'dblp.uni-trier.de', 'landmark.cs.cornell.edu']
2017-03-23 20:30:50 [anansi.dao.frontier] INFO: Dequeuing batch of frontier URIs; frontier size 905718; select using sample {TABLESAMPLE BERNOULLI(0.11040964185320375)}, dominant {AND uri NOT LIKE '%infocenter.arm.com%' AND uri NOT LIKE '%aclanthology.info%' AND uri NOT LIKE '%events.cornell.edu%' AND uri NOT LIKE '%www.aaai.org%' AND uri NOT LIKE '%digitalcommons.unl.edu%' AND uri NOT LIKE '%dblp.uni-trier.de%' AND uri NOT LIKE '%landmark.cs.cornell.edu%'}
2017-03-23 20:30:50 [anansi.dao.frontier] INFO: Populated cache with 614 frontier URIs
2017-03-23 20:30:59 [scrapy.extensions.logstats] INFO: Crawled 5616 pages (at 468 pages/min), scraped 276 items (at 45 items/min)
2017-03-23 20:31:46 [anansi.dao.frontier] INFO: ['infocenter.arm.com', 'aclanthology.info', 'events.cornell.edu', 'landmark.cs.cornell.edu', 'www.aaai.org', 'digitalcommons.unl.edu']
2017-03-23 20:31:46 [anansi.dao.frontier] INFO: Dequeuing batch of frontier URIs; frontier size 903676; select using sample {TABLESAMPLE BERNOULLI(0.11065913004218327)}, dominant {AND uri NOT LIKE '%infocenter.arm.com%' AND uri NOT LIKE '%aclanthology.info%' AND uri NOT LIKE '%events.cornell.edu%' AND uri NOT LIKE '%landmark.cs.cornell.edu%' AND uri NOT LIKE '%www.aaai.org%' AND uri NOT LIKE '%digitalcommons.unl.edu%'}
2017-03-23 20:31:46 [anansi.dao.frontier] INFO: Populated cache with 634 frontier URIs
2017-03-23 20:31:59 [scrapy.extensions.logstats] INFO: Crawled 6166 pages (at 550 pages/min), scraped 300 items (at 24 items/min)