Skip to content

Instantly share code, notes, and snippets.

@dangra
Created May 8, 2014 19:33
Show Gist options
  • Save dangra/bc710c999c0b6bc6c367 to your computer and use it in GitHub Desktop.
Save dangra/bc710c999c0b6bc6c367 to your computer and use it in GitHub Desktop.
$ scrapy shell https://www.ssehl.co.uk/HALO/publicLogon.do -c "response.xpath('//title').extract()"
2014-05-08 16:33:22-0300 [scrapy] INFO: Scrapy 0.23.0 started (bot: scrapybot)
2014-05-08 16:33:22-0300 [scrapy] INFO: Optional features available: ssl, http11
2014-05-08 16:33:22-0300 [scrapy] INFO: Overridden settings: {'LOGSTATS_INTERVAL': 0}
2014-05-08 16:33:22-0300 [scrapy] INFO: Enabled extensions: TelnetConsole, CloseSpider, WebService, CoreStats, SpiderState
2014-05-08 16:33:22-0300 [scrapy] INFO: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, RedirectMiddleware, CookiesMiddleware, ChunkedTransferMiddleware, DownloaderStats
2014-05-08 16:33:22-0300 [scrapy] INFO: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware
2014-05-08 16:33:22-0300 [scrapy] INFO: Enabled item pipelines:
2014-05-08 16:33:22-0300 [scrapy] DEBUG: Telnet console listening on 0.0.0.0:6023
2014-05-08 16:33:22-0300 [scrapy] DEBUG: Web service listening on 0.0.0.0:6080
2014-05-08 16:33:22-0300 [default] INFO: Spider opened
2014-05-08 16:33:25-0300 [default] DEBUG: Crawled (200) <GET https://www.ssehl.co.uk/HALO/publicLogon.do> (referer: None)
[u'<title>Public Notices</title>']
diff --git a/scrapy/core/downloader/contextfactory.py b/scrapy/core/downloader/contextfactory.py
index e20830c..2eaa3eb 100644
--- a/scrapy/core/downloader/contextfactory.py
+++ b/scrapy/core/downloader/contextfactory.py
@@ -10,7 +10,8 @@ class ScrapyClientContextFactory(ClientContextFactory):
def __init__(self):
# see this issue on why we use TLSv1_METHOD by default
# https://github.com/scrapy/scrapy/issues/194
- self.method = SSL.TLSv1_METHOD
+ #self.method = SSL.TLSv1_METHOD
+ self.method = SSL.SSLv3_METHOD
def getContext(self, hostname=None, port=None):
ctx = ClientContextFactory.getContext(self)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment