6:30 - 6:40: settling in
6:40 - 7:00: Julian, quick introduction to PyPy
7:00 - 7:10: Andy, quick introduction to CLI
7:10 - 7:15: James, NYC Python announcements
7:15 - 8:30: James, CPython workshop
8:30 - 9:00: mingling + sponsor announcements
CPython for greater understanding of the Python programming language (but "reference implementations always overspecify") Reading source to solve problems Getting involved, contributing to the project
This workshop will cover the basics of the CPython runtime and interpreter. There is an enormous amount of material to cover, and I'll try to to rush through as much as I can.
This introduction will be rushed, so it may not be perfectly accurate.
CPython is the reference implementation of the Python programming language. The Python programming language exists separate from this implementation, and, in fact, alternate implementations of the language exist. Julian spoke about PyPy. There are also implementations built on top of the .NET runtime (IronPython,) the JVM (Jython,) &c.
CPython is still the dominant, most commonly used implementation of the Python language. (Whether this should or should not be the case is an independent question, one which we can see Julian feels passionately about.) CPython is probably what you are using when you type python at the command line.
CPython is written in the C programming language. It should compile without errors on a C89 or C99 compliant compiler.
If you want to learn more about the C programming language, stop by office hours. There are many C/C++/Java programmers who might be able to help you out. There are also on-line resources such as c.learncodethehardway.org/book. You may find the syntax of C to be not completely alien to you as a Python programmer. The CPython interpreter is written to be fairly straightforward to understand and to contribute to.
We are going to be using a couple of other tools, too. autoconf, gcc, gdb, coreutils.
We recommend using our Ubuntu virtual machine (locally or remotely,) since it removes a lot of the difficulty from this process.
These tools are extremely rich, and we could do a workshop on each of them individually. We're going to use them in this workshop, but going into depth on anything outside of basic gdb is out of our scope.
We're going to start this workshop by downloading the C source code for CPython 3.3.3. We're going to build it, install it, and run it under gdb.
http://python.org/download/
wget 'http://python.org/ftp/python/3.3.3/Python-3.3.3.tar.xz' # get the source code
tar xJvf Python-3.3.3.tar.xz # extract
# for convenience
sudo apt-get install build-essential
sudo apt-get build-dep python3.3 # gives you all the other software Python depends on
# configure, build, install
cd Python-3.3.3
CFLAGS="-g4 -ggdb -gdwarf-4" ./configure --with-pydebug --prefix=$PWD-build
make -j9
make install
Doc # reStructuredText documentation
Grammar/Grammar # Python 'grammar' EBNF
Include # header files
Lib # the Python standard library, Python code, e.g., collections.namedtuple
Modules # Python modules written in C, math.pow
Objects # Python basic types
Parser # language parser
Python # interpreter
$PWD-build/bin/python3
def f(x, y):
x, y = y, x
>>> from dis import dis
>>> dis(f)
2 0 LOAD_FAST 1 (y)
3 LOAD_FAST 0 (x)
6 ROT_TWO
7 STORE_FAST 0 (x)
10 STORE_FAST 1 (y)
13 LOAD_CONST 0 (None)
16 RETURN_VALUE
$ find -iname '*.c' -print0 | xargs -0 grep ROT_TWO
./Python/compile.c: case ROT_TWO:
./Python/compile.c: ADDOP(c, ROT_TWO);
./Python/compile.c: ADDOP(c, ROT_TWO);
./Python/peephole.c: codestr[i] = ROT_TWO;
./Python/peephole.c: codestr[i+1] = ROT_TWO;
./Python/ceval.c: TARGET(ROT_TWO)
gdb --args $PWD-build/bin/python3
(gdb) source Tools/gdb/libpython.py
(gdb) r
>>> def f(x, y):
... x, y = y, x
^C
(gdb) tbreak ceval.c:1389
(gdb) c
>>> f(10, 20)
(gdb) list
(gdb) info macro TARGET
(gdb) info function PyObject_GetAttr
(gdb) backtrace
(gdb) next
(gdb) step
(gdb) finish
class Foo(int):
def __mul__(self, other):
print('Foo.__mul__({}, {})'.format(self, other))
class Bar(int):
def __mul__(self, other):
print('Bar.__mul__({}, {})'.format(self, other))
foo, bar = Foo(10), Bar(20)
foo * bar
bar * foo
10 * foo
bar * 10
class Foo(int):
def __mul__(self, other):
print('Foo.__mul__({}, {})'.format(self, other))
def __rmul__(self, other):
print('Foo.__rmul__({}, {})'.format(self, other))
class Bar(int):
def __mul__(self, other):
print('Bar.__mul__({}, {})'.format(self, other))
def __rmul__(self, other):
print('Bar.__rmul__({}, {})'.format(self, other))
foo, bar = Foo(10), Bar(20)
foo * bar
bar * foo
10 * foo
bar * 10
class Foo(int):
def __mul__(self, other):
print('Foo.__mul__({}, {})'.format(self, other))
def __rmul__(self, other):
print('Foo.__rmul__({}, {})'.format(self, other))
class Bar(Foo):
def __mul__(self, other):
print('Bar.__mul__({}, {})'.format(self, other))
def __rmul__(self, other):
print('Bar.__rmul__({}, {})'.format(self, other))
foo, bar = Foo(10), Bar(20)
foo * bar
bar * foo
10 * foo
bar * 10
from dis import dis
def f(x, y):
return x * y
dis(f)
Note If the right operand’s type is a subclass of the left operand’s type and that subclass provides the reflected method for the operation, this method will be called before the left operand’s non-reflected method. This behavior allows subclasses to override their ancestors’ operations.
$ python3
hash(100)
hash(10)
hash(2)
hash(1)
hash(-100)
hash(-10)
hash(-2)
hash(-1)
$ pypy
hash(-1)
tb ceval.c:2671
p PyCFunction_Check(func)
class Foo(object):
def __hash__(self):
return -1
tb builtin_hash
Objects/abstract.c vs Objects/object.c vs Objects/typeobject.c
p ((PyFunctionObject*)(((PyMethodObject*)(func))->im_func))->func_name
docs.python.org/devguide pythonmentors.com
def dec(func):
return func
@dec
def foo(x, y):
return x * y
dec = lambda func: func
@(lambda func:func)
def foo(x, y):
return x * y
http://mail.python.org/pipermail/python-dev/2004-August/046711.html
Grammar/Grammar
-decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE
+decorator: '@' testlist NEWLINE
Python/ast.c
-name_expr = ast_for_dotted_name(c, CHILD(n, 1));
+name_expr = ast_for_testlist(c, CHILD(n, 1));
make test
Lib/test/test_decorators.py
Lib/test/test_parser.py
- validate_dotted_name(CHILD(tree, 1)) &&
+ validate_testlist(CHILD(tree, 1)) &&
bugs.python.org
Decorator syntax currently allows only a dotted_name after the @. As far as I can tell, this was a gut-feeling decision made by Guido. [1]
I spoke with Nick Coghlan at PyTexas about this, and he suggested that if someone did the work, there might be interest in revisiting this restriction.
The attached patch allows any testlist to follow the @.
The following are now valid:
@(lambda x:x)
def f():
pass
@(spam if p else eggs)
def f():
pass
@spam().ham().eggs()
def f():
pass
[1] http://mail.python.org/pipermail/python-dev/2004-August/046711.html