switchcase-presentation.org

switchcase-presentation.org

Implementing switch-case in python

Background

What happens when you run Python code?

It sorta feels like this

{{{imglink(https://media.giphy.com/media/S5qhrBEfPQHFS/giphy.gif)}}}

Let’s be honest, we’ve all felt like this…

However it can sometime feel like this

{{{imglink(https://media.giphy.com/media/l3q2ENXbfEEHfMpLG/giphy.gif)}}}

Python is so dynamic that we sometimes don’t know what anything is

{{{imglink(https://i.chzbgr.com/full/5072733696/hED055FEE/sup-snake)}}}

How do we better understand Python so we can be more free?

Exploring Python Internals

{{{imglink(https://media1.tenor.com/images/6830c3545c23a5c95640af1023880c8d/tenor.gif)}}}

What better way to explore than adding a new feature?

What is Python?

• A fun programming language
• Interpreted, Object oriented, and high-level language
• Simple and easy to learn syntax
• Productive on day 1
• Python is a language specification
  • Multiple different implementation
    • {{{color(green, CPython)}}}
    • Jython
    • PyPy
    • Cython

What is switch-case

A language statement allowing for multiway conditional branches.

switch (n)
{
    case 1: // code to be executed if n = 1;
        break;
    case 2: // code to be executed if n = 2;
        break;
    default: // code to be executed if n doesn't match any cases
}

An alternative to if/elif/else (from python) to execute different sections of code

Why isn’t this already in python?

• Python core does not include switch-case statements
• This is because of PEP-0275 and PEP-3103 which reject the inclusion of such statements.
• All you need to know about possible syntax, implementation methods, and an example are in the PEPS
• Fun Fact: The cause for this decision was from a poll in a 2007 PyCon keynote

Is it possible?

ABSOLUTELY!

Is it easy?

…

What is CPython

• The magic snake that lives your computer
• CPython is the python binary you probably have on your system right now
• The “C” implementation of the language
• Interprets Compiled .py code and “executes” it on your machine
  1. {{{color(green, Parse Tree Generation)}}}
  2. {{{color(green, AST Generation)}}}
  3. {{{color(green, Bytecode Generation)}}}
  4. Bytecode Optimization
  5. Code Object Generation
  6. Code Object Execution

Picture!

{{{imglink(http://image.slidesharecdn.com/suhasastpycon2016-160925142741/95/hacking-the-python-ast-10-638.jpg)}}}

Grammar

• Python uses Extend Backus-Naur Form (EBNF) grammer to describe all statements through rules

if_stmt: 'if' namedexpr_test ':' suite ('elif' namedexpr_test ':' suite)* ['else' ':' suite]
while_stmt: 'while' namedexpr_test ':' suite ['else' ':' suite]
for_stmt: 'for' exprlist 'in' testlist ':' [TYPE_COMMENT] suite ['else' ':' suite]

All these rules are used to generate tokens which will verify our python code

Some cpython tokens

Parsing 1

import parser, pprint
code_str = """def hello():
                  return "hello!"
           """
st = parser.suite(code_str)
pprint.pprint(parser.st2list(st))

Parsing 2

AST

• This raw parse tree (ST) is an intermediate representation which ensures syntactic correctness
• Now we need to clean up this tree removing unnecessary tokens such as NEWLINE or COLON
• This is where we construct the Abstract Syntax Tree (AST)
• The AST will start represent a form we will recognize

FunctionDef(identifier name, arguments args,
            stmt* body, expr* decorator_list, expr? returns,
            string? type_comment)

Parsing 3

import subprocess;subprocess.call(['pip', 'install', 'astpretty', '--user'])
import ast
import astpretty
code_str = """def hello():
                  return "hello!"
           """
node = ast.parse(code_str, mode="exec")

print(astpretty.pformat(node))

Parsing 4

Compiler

• Despite what you think, python is compiled
• The AST gets transformed into opcodes / bytecode
• opcodes are then fed into the python virtual machine
• A group of opcodes is called a block
• An opcode in a block can jump to other block
• Later python will interpret a block

dis

The python built in module dis can show us compiled opcode output from python code

import dis
def hello():
    print("i'm in the function")
    return "hello!"
dis.dis(hello)

  3           0 LOAD_GLOBAL              0 (print)
              2 LOAD_CONST               1 ("i'm in the function")
              4 CALL_FUNCTION            1
              6 POP_TOP

:

  4           8 LOAD_CONST               2 ('hello!')
             10 RETURN_VALUE

 |           | |                        | |
	|      	    | |                        | |
	      |           | +---------o Opcode name  | |
	      |           |                          | +----o human interpretation
	      |           +-----o Bytecode Address   |
	      |                                      |
	      +--------o Line number                 +--o Argument (if any)

Execution

After all the parsing and compiling we finally get the output we are looking for

def hello():
    return "hello!"
print(hello())

hello!

Implementing Switch Case

What does switch-case look like?

There are several paths and looks we can take when implementing switch-case.

I chose a syntax that looks most familiar to other languages

a = 0
switch a:
    case 1:
        print("yay")
    case 2:
        print("woo")
    else:
        print("bleh")

blehh

Step 1: Modify the grammar

Grammar/Grammar

switch_stmt:
'switch' namedexpr_test ':' NEWLINE
INDENT ('case' namedexpr_test ':' suite)*
       ['else' ':' suite]
DEDENT

Step 2: Modify the ASDL (AST definition)

Switch(expr test, casehandler* cases, stmt* orelse)
casehandler = CaseHandler(expr test, stmt* body)
              attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset)

With these definitions in place (Grammer + ASDL), CPython generates C code for us.

We can use that C code to implemente switch case logic

Step 3: Modify the AST

static stmt_ty
ast_for_switch_stmt(struct compiling *c, const node *n)
{
    /* This function builds the ast for the 'switch' 'case' statements
       (づ￣ ³￣)づ

       switch_stmt: 'switch' namedexpr_test ':'
       ('case' namedexpr_test  (',' namedexpr_test)*  ':' suite)+
       ['else' : suite]
    */

• Update Python/ast.c to parse switch-case
• Very similar to manipulating an array of AST nodes
• But we still can’t execute it. :(

Step 4: Modify the Compiler

static int
compiler_switch(struct compiler *c, stmt_ty s)

In this function we turn our switch-case AST into bytecode

Bytecode output

import dis
def onomatopoeia():
    switch "woohoo":
        case "whippie":
            print("whippie case")
    return
dis.dis(onomatopoeia)

3           0 LOAD_CONST               1 ('woohoo')

4           2 DUP_TOP
            4 LOAD_CONST               2 ('whippie')
            6 COMPARE_OP               2 (==)
            8 POP_JUMP_IF_FALSE       22
           10 POP_TOP

5          12 LOAD_GLOBAL              0 (print)
           14 LOAD_CONST               3 ('whippie case')
           16 CALL_FUNCTION            1
           18 POP_TOP
           20 JUMP_FORWARD             4 (to 26)
      >>   22 POP_TOP
           24 JUMP_FORWARD             0 (to 26)

6     >>   26 LOAD_CONST               0 (None)
           28 RETURN_VALUE

Sample code

a = "woop"
switch type(a):
    case float:
        print("it's a float")
    case int:
        print("it's a int")
    case str:
        print("it's a str")
    else:
        print("it's a something ... ")

it's a str

Takeaways

{{{imglink(https://i.imgur.com/AOyHkU6.gif)}}}

• CPython is VERY hackable
• Building your own super language requires very little code
• Appreciate the dynamic and flexible nature of python

Useful links

• EliBendersky-python-internals
• Inside the Python Virtual Machine
• CPython Bytecode Compiler is Dumb