Testing PyPI downloads (see https://mail.python.org/pipermail/distutils-sig/2013-November/023127.html)

test_pypi_downloads.py

#!/usr/bin/python
import hashlib, time
try:
    from urllib.request import urlopen
except ImportError:
    from urllib import urlopen

url = 'https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.10.1.tar.gz'
md5sum_expected = '3a04aa2b32c76c83725ed4d9918e362e'
response = urlopen(url)
##import pdb; pdb.set_trace()
socket = getattr(response.fp, 'raw', response.fp)._sock

# If you set this to False, you get SSLError exceptions.
# If you set this to True, you get truncated downloads that fail md5sum checks.
socket.suppress_ragged_eofs = False

data = b''
while True:
    chunk = socket.recv(8192)
    print("%d %d" % (len(chunk), len(data) + len(chunk)))
    if not chunk:
        break
    data += chunk
md5sum_actual = hashlib.md5(data).hexdigest()
filename = 'r%d' % time.time()
if md5sum_actual == md5sum_expected:
    print("OK")
    open(filename + '.ok', 'wb').write(data)
else:
    print("BAD: %s" % md5sum_actual)
    open(filename + '.bad', 'wb').write(data)
print("downloaded %d bytes" % len(data))

TIL: github doesn't send me emails when people comment on my gists. :(

So if I understand this correctly, @rgbkrk and @smashwilson saw this error on Windows, but outside the Rackspace network? IOW the issue is on the software side?

I'm also curious about why the ssl module suppresses this SSLError by default. @smashwilson, @rgbkrk, if you change socket.suppress_ragged_eofs in this script to True, do you see MD5 checksum errors (i.e. 'BAD: ...' messages)?

BTW another user also could reproduce this on Rackspace Cloud: https://mail.python.org/pipermail/distutils-sig/2013-December/023324.html