Fun times with simplejson on #python-dev

python-dev.log

[2015-01-21 16:12:19] <ionelmc> haypo: it only happens when i got simplejson installed :-)
[2015-01-21 16:13:14] <haypo> ionelmc: so you know the fix, uninstall simplejson :-D
[2015-01-21 16:13:22] <haypo> ionelmc: it looks like a bug in this module
[2015-01-21 16:13:39] <haypo> ionelmc: python 2.7 provides a json module which is based on simplejson
[2015-01-21 16:13:51] <haypo> ionelmc: why not using the "builtin" json module?
[2015-01-21 16:14:08] <ionelmc> cauze someone was too lazy to cleanup the imports
[2015-01-21 16:14:19] <ionelmc> :-)
[2015-01-21 16:14:42] <ionelmc> haypo: so no speed difference?
[2015-01-21 16:14:50] <ionelmc> that someone is also too lazy to test :)
[2015-01-21 16:15:08] <Taggnostr2> ionelmc, in 2.7 the json module is accelerated in C, so it should be fast
[2015-01-21 16:15:56] <haypo> ionelmc: it comes with a C implementation, _json
[2015-01-21 16:16:08] <ionelmc> but now i have this morbid curiosity to know why simplejson segfaults ;-\
[2015-01-21 16:16:09] <haypo> IMO _json is full of bugs :)
[2015-01-21 16:16:16] <haypo> a lot of bugs have already been fixed
[2015-01-21 16:16:32] <Taggnostr2> maybe they haven't been fixed upstream in simplejson
[2015-01-21 16:17:19] <haypo> https://hg.python.org/cpython/log/2cd0aa44d53c/Modules/_json.c : history of the file in Python 2.7
[2015-01-21 16:17:46] <haypo> ionelmc: ^^ see the long list of "fix ... segfault"
[2015-01-21 16:18:03] — ionelmc weeps ...
[2015-01-21 16:18:48] <haypo> Taggnostr2: oh. i don't know the relation between cpython and simplejson
[2015-01-21 16:19:10] <Taggnostr2> haypo, afaik they are being developed separately
[2015-01-21 16:19:20] <haypo> https://github.com/simplejson/simplejson/commits/master doesn't look to be dead
[2015-01-21 16:19:40] <haypo> i heard that https://pypi.python.org/pypi/ujson is much faster
[2015-01-21 16:19:43] <Taggnostr2> sometime someone checks simplejson and see if there's something that should be ported to our json module
[2015-01-21 16:20:04] <Taggnostr2> and I guess the opposite also happens, but it's not systematic
[2015-01-21 16:34:26] <storchaka> What issues with json or simplejson?
[2015-01-21 16:34:44] <haypo> storchaka: sometimes it crashs :)
[2015-01-21 16:34:54] <haypo> storchaka: ionelmc uses python 2.7.3 from ubuntu 12.04
[2015-01-21 16:35:10] <haypo> storchaka: there is also a question if simplejson retrieved fixes from cpython
[2015-01-21 16:35:12] <storchaka> haypo: How it can be reproduced?
[2015-01-21 16:35:32] <ionelmc> haypo: regarding ujson, i found this gem today http://mod16.org/hurfdurf/?p=266
[2015-01-21 16:35:40] <haypo> storchaka: the bug is in simplejson, not in json
[2015-01-21 16:35:46] <storchaka> simplejson retrieved fixes from cpython
[2015-01-21 16:35:49] <ionelmc> looks like it's really borken on the inside
[2015-01-21 16:35:59] <Wooble> I was under the impression the stdlib json *was* simplejson.
[2015-01-21 16:36:09] <haypo> Wooble: yes and no
[2015-01-21 16:36:31] <haypo> Wooble: there are two source code repositories, https://github.com/simplejson/simplejson/ & https://hg.python.org/cpython/
[2015-01-21 16:36:31] <storchaka> May be not all or may be Ubuntu 12.04 uses too old simplejson
[2015-01-21 16:36:56] <ionelmc> storchaka: i'm using the latest from pypi
[2015-01-21 16:37:03] <Wooble> Oh, I guess the codebases have diverged since it was incorporated.
[2015-01-21 16:38:06] <storchaka> ionelmc: Please file a report on simplejson bugtracker
[2015-01-21 16:38:12] <haypo> Wooble: correct
[2015-01-21 16:38:12] <ionelmc> already did
[2015-01-21 16:38:37] <ionelmc> https://github.com/simplejson/simplejson/issues/114
[2015-01-21 16:41:45] <storchaka> Thanks. It may be a bug in Python 2.7.3 fixed in 2.7.9
[2015-01-21 16:42:18] <haypo> storchaka: yes
[2015-01-21 16:43:16] <haypo> ionelmc: oh funny, pip crashs while installing simplejson?
[2015-01-21 16:48:02] <ionelmc> haypo: during the second install
[2015-01-21 16:49:02] <ionelmc> haypo: different stack trace now
[2015-01-21 16:49:04] <ionelmc> really strange
[2015-01-21 16:49:15] <haypo> ionelmc: multiple crashs have been fixed ;)
[2015-01-21 16:49:35] <ionelmc> heh
[2015-01-21 16:54:22] <ionelmc> haypo: does the debug build affect memory layout in any way?
[2015-01-21 16:55:17] <ionelmc> also, with the minimal testcase, i get no frames from faulthandler
[2015-01-21 16:55:22] <haypo> ionelmc: i have no idea. i didn't fix json bugs
[2015-01-21 16:55:41] <ionelmc> haypo: i suppose cause the segfault now comes from Py_Finalize
[2015-01-21 16:56:04] <haypo> ionelmc: gdb should tell you where you are
[2015-01-21 16:56:25] <haypo> ionelmc: yes, during early python initialization and during python finalization, there is no frame
[2015-01-21 16:56:33] <haypo> ionelmc: or faulthandler failed to retrieve the current frame
[2015-01-21 16:58:49] <haypo> ionelmc: i tried your testcase, i see in gdb a crash in PyRun_SimpleFileExFlags > PyErr_PrintEx > ... > Py_Finalize > PyGC_Collect > ... (crash)
[2015-01-21 16:59:23] <ionelmc> haypo: refcount bug? :)
[2015-01-21 16:59:41] <haypo> ionelmc: if you really want to make the world better, compare _json.c with the file in simplejson, and merge back fixes in simplejson
[2015-01-21 16:59:55] <ionelmc> how the hell you got it to reproduce?
[2015-01-21 17:00:35] <haypo> using your testcase, https://github.com/simplejson/simplejson/issues/114