Skip to content

Instantly share code, notes, and snippets.

@coolreader18
Last active March 30, 2024 08:05
Show Gist options
  • Save coolreader18/6dbe0be2ae2192e90e1a809f1624c694 to your computer and use it in GitHub Desktop.
Save coolreader18/6dbe0be2ae2192e90e1a809f1624c694 to your computer and use it in GitHub Desktop.
CPython segfault in 5 lines of code
class E(BaseException):
def __new__(cls, *args, **kwargs):
return cls
def a(): yield
a().throw(E)

CPython Segfault in 5 lines of code

(Works as described on at least CPython 3.6-3.8)

So this is pretty weird, right?

Let's look at why this happens. First, we define a subclass of Exception that returns a non-exception from its __new__ constructor. This could be anything, as long as it's not actually an instance of BaseException:

import random
class E(BaseException):
    def __new__(cls, *args, **kwargs):
        return random.choice([1, (), {}, ""])
def a(): yield
# will still always segfault:
a().throw(E)

Well, almost anything; a few things I've found that don't segfault are lists and generators- I'll come back to this later.

Then, we call the generator.throw() with E as its only argument. gen.throw() accepts up to 3 arguments, which mirror the arguments you'd pass to raise in Python 2:

raise TypeError, "Couldn't do thing", other_exc.__traceback__

We only pass it one argument, which because it's a type object (and a subclass of BaseException) it calls to get an exception object to throw to the generator (again, like raise). Usually, of course, type objects return an instance of themselves when called, but this one we've defined doesn't. But shouldn't it still just TypeError? And does raise do the same thing?

class E(BaseException):
    def __new__(cls, *args, **kwargs):
        return cls
# Python 3 still supports `raise ExceptionType`, just not the other arguments
raise E
Traceback (most recent call last):
  File "segfault.py", line 8, in <module>
    raise E
TypeError: calling <class '__main__.E'> should have returned an instance of BaseException, not <class 'type'>

raise works fine. Hmmm. Let's try looking up that error message in the CPython codebase.

// ceval.c, line 4238
    /* We support the following forms of raise:
       raise
       raise <instance>
       raise <type> */

     /* VVVVVVVVVVVVVVVVVVVVVVVVVVV is the argument a subclass of BaseException? */
    if (PyExceptionClass_Check(exc)) {
        type = exc;
     /* VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV call the type object */
        value = _PyObject_CallNoArg(exc);
        if (value == NULL)
            goto raise_error;
          /* VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV type check for instance of BaseException */
        if (!PyExceptionInstance_Check(value)) {
            _PyErr_Format(tstate, PyExc_TypeError,
                        /* VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV search result for error message */
                          "calling %R should have returned an instance of "
                          "BaseException, not %R",
                          type, Py_TYPE(value));
             goto raise_error;
        }
    }
    else if (PyExceptionInstance_Check(exc)) {
        /* most common path, raising an already constructed exception */
        value = exc;
        type = PyExceptionInstance_Class(exc);
        Py_INCREF(type);
    }
    /* ... */

It looks like this type checking is done specifically in the code for executing the RAISE_VARARGS bytecode instruction, and not in any deeper machinery in the CPython interpreter. So does generator.throw() do this?

// genobject.c, line 483
     /* VVVVVVVVVVVVVVVVVVVVVVVVVVV check if it's a subclass of BaseException... */
    if (PyExceptionClass_Check(typ))
        PyErr_NormalizeException(&typ, &val, &tb);

          /* VVVVVVVVVVVVVVVVVVVVVVVVVVVVVV common path again */
    else if (PyExceptionInstance_Check(typ)) {
        /* Raising an instance.  The value should be a dummy. */
        /* ... */

Nope, it just passes it off to the interpreter function PyErr_NormalizeException. Does that validate the result of calling the exception type? Well, the source is pretty long and not that interesting, but you can read it for yourself and see that it does no checks whatsoever.

So that's why the segfault happens: CPython places a non-exception PyObject as the current exception for the thread, and when it tries to do an operation on it (like editing the traceback), it reaches past the size of the PyTupleObject struct that's actually stored there and segfaults.

I'm still not sure why lists and generators don't cause segfaults; it's not because they conform to the sequence protocol as strings and tuples do as well, and tuples and lists are usually very similar in functionality (besides, like, mutability). I'd assume that it's something in the interpreter code that catches that an exception value isn't actually an exception (but only for these types) and throws a proper error, but I'm not sure. If anyone does actually know, I'd love to hear about it.

Note: I came across this while trying to figure out the proper behavior for generator.throw() for RustPython. I suppose the proper behavior in order to be truly CPython compliant is to just dereference a null pointer! 😁

PyBaseExceptionRef::try_from_object(res).unwrap_or_else(|_| unsafe { *std::ptr::null() })

BPO issue

See discussion on HN or Reddit

@Hellkyte
Copy link

Funnily enough this works as expected in cpython2.7.

@coolreader18
Copy link
Author

Yeah, some other people have also said it doesn't happen for them on other versions (or only as a script, not in the REPL). I've updated the post to include the version range it occurs in.

@eric-wieser
Copy link

@coolreader18: If you haven't already, please file a bug at https://bugs.python.org

@sekrause
Copy link

An issue has been created on BPO: https://bugs.python.org/issue39091

@divinity76
Copy link

same in cygwin with 3.6.9,

$ python3 --version
Python 3.6.9
$ python3 -c "(i for i in []).throw(type('E', (BaseException,), dict(__new__=lambda cls, *args: cls))())"
Segmentation fault (core dumped)

neat

@P3GLEG
Copy link

P3GLEG commented Dec 19, 2019

Wow man! Nice job that's an awesome find.

in /tmp
$ uname -s
Darwin

in /tmp 
$ python --version
Python 3.7.5

in /tmp
$ python segfault.py
[1]    87070 segmentation fault  python segfault.py

@pauldraper
Copy link

pauldraper commented Dec 19, 2019

67 chars

$ python3.7 -VV
Python 3.7.5 (default, Nov  7 2019, 10:50:52) 
[GCC 8.3.0]
$ python3.7 -c "(i for i in[]).throw(type('',(IOError,),{'__new__':lambda a,*b:a}))"
Segmentation fault (core dumped)

@miqueet
Copy link

miqueet commented Dec 19, 2019

same on python 3.6

$ python3 -c "(i for i in[]).throw(type('',(IOError,),{'__new__':lambda a,*b:a}))"
Segmentation fault (core dumped)
$ python3 -V
Python 3.6.8

@bstaletic
Copy link

This is fun!

% python3.5 -c "(i for i in []).throw(type('', (IOError,), dict(__new__=lambda cls, *args: cls))())"
zsh: segmentation fault  python3.5 -c
% python3.5 -V
Python 3.5.9

@horvatha
Copy link

@bl-ue
Copy link

bl-ue commented Jun 17, 2021

Hey @coolreader18 — what's going on here? It looks like you fixed this, but in Python 3.9.5 it's still segfaulting:

python3
Python 3.9.5 (default, May  4 2021, 03:36:27) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class E(BaseException):
...     def __new__(cls, *args, **kwargs):
...         return cls
... 
>>> def a(): yield
... 
>>> 
>>> a().throw(E)
zsh: segmentation fault  python3

Edit: I see, python/cpython#17658 is not merged yet. 🤔

@dheeman00
Copy link

I am constantly getting segmentation fault from a CPython script. I used the same script on a different machine and it worked perfectly but now I am getting that error every time I am running that script. Can anyone share some suggestions.

@nocturn9x
Copy link

(() for _ in []).throw(type("E", (BaseException, ), {"__new__": lambda *_: 1}))

This will segfault too ;)

@DavidBuchanan314
Copy link

DavidBuchanan314 commented Aug 27, 2021

btw, here's my segfault oneliner (which is unrelated to this issue):

eval((lambda:0).__code__.replace(co_consts=()))

@fanninpm
Copy link

This particular segfault is fixed in CPython 3.9.7, released today (30 August 2021).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment