Skip to content

Instantly share code, notes, and snippets.

@njsmith
Last active August 29, 2015 14:07
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save njsmith/0651d1d3266bf017c9e2 to your computer and use it in GitHub Desktop.
Save njsmith/0651d1d3266bf017c9e2 to your computer and use it in GitHub Desktop.
workaround python 2/3 incompatibilities in formatting/parsing of long integers
import six
import tokenize
import numpy as np
def _filter_tokens(s):
last_token_was_number = False
for token in tokenize.generate_tokens(six.StringIO(s).read):
token_type = token[0]
token_string = token[1]
if (last_token_was_number
and token_type == tokenize.NAME
and token_string == "L"):
continue
else:
yield token
last_token_was_number = (token_type == tokenize.NUMBER)
def fixup_header(s):
return tokenize.untokenize(_filter_tokens(s))
def test_fixup_header():
assert (np.safe_eval(fixup_header('{"a": 1L, "b": "1L"}'))
== {"a": 1, "b": "1L"})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment