py3-difficulties.md

Known py3 difficulties:

• https://bugs.python.org/issue22233

  py3 apparently can't speak HTTP.

• https://github.com/openstack/swift/blob/2.9.0/swift/common/wsgi.py#L145

  Need to change the default content-type to not be text/plain. Note that mimetools no longer exists under py3.

• https://www.python.org/dev/peps/pep-3333/#unicode-issues

  On Python platforms where the str or StringType type is in fact Unicode-based (e.g. Jython, IronPython, Python 3, etc.), all "strings" referred to in this specification must contain only code points representable in ISO-8859-1 encoding (\u0000 through \u00FF, inclusive). It is a fatal error for an application to supply strings containing any other Unicode character or code point.

  ... but swift requires UTF-8 headers for accounts/containers. So we'll need to intentionally mangle a whole lot of strings. And we'll need to worry about whether third-party middlewares are using utf-8 or utf-8-masquerading-as-latin-1 strings.

• https://github.com/openstack/swift/blob/2.9.0/swift/proxy/server.py#L220

  socket._fileobject isn't a thing under py3. Maybe it'd work to set eventlet.wsgi.HttpProtocol.rbufsize / eventlet.wsgi.HttpProtocol.wbufsize instead?

• https://github.com/openstack/swift/blob/2.9.0/swift/common/bufferedhttp.py#L51-L54

  socket._socketobject isn't a thing under py3.

https://bugs.python.org/issue22233 just landed (2 weeks ago) on October 7

py3 random implementation is different than in py2. This causes tests that rely on setting the random seed and having deterministic behavior aren't portable across py2 to py3. (see patch chain associated with https://review.openstack.org/#/c/401397/)

bytes vs native strings in headers (related to latin-1/utf8 encoding issues) to ensure "b'some string'" doesn't get sent back in header values

rings use pickle, which can have issues across py2 and py3 (https://bugs.launchpad.net/swift/+bug/1644387)

all existing metadata is pickled, so the previous py2/py3 pickle issues must be resolved

We currently require that delimiter queries be a single byte and decodable as UTF-8; as a result, we essentially limit ourselves to ASCII. Do we continue with this, or change the requirement to be a single codepoint?

Bankers' rounding will probably result in some flaky tests:

======================================================================
FAIL: test_greater_with_offset (test.unit.common.test_utils.TestTimestamp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/vagrant/swift/test/unit/common/test_utils.py", line 632, in test_greater_with_offset
    timestamp, value, value))
nose.proxy.AssertionError: False is not true : 1495139343.99161_0000000000000001 is not greater than '1495139343.991615' given '1495139343.991615'
    "False is not true : 1495139343.99161_0000000000000001 is not greater than '1495139343.991615' given '1495139343.991615'" = self._formatMessage("False is not true : 1495139343.99161_0000000000000001 is not greater than '1495139343.991615' given '1495139343.991615'", "%s is not true" % safe_repr(False))
>>  raise self.failureException("False is not true : 1495139343.99161_0000000000000001 is not greater than '1495139343.991615' given '1495139343.991615'")