Created
October 15, 2019 20:10
-
-
Save dabeaz/7d8838b54dba5006c58a40fc28da9d5a to your computer and use it in GitHub Desktop.
PyCon India 2019, Code from Keynote Presentation by @dabeaz
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Code from PyCon India 2019 Keynote Talk | |
David Beazley (https://www.dabeaz.com) | |
====================================== | |
This code is presented "as is" and represents what was live-coded | |
during my closing keynote presentation at PyCon India, Chennai, | |
October 13, 2009. I have made no changes to the files. | |
Requires: Python 3.6+, numpy, pygame | |
Also Requires: https://github.com/aochagavia/rocket_wasm/blob/master/html/program.wasm | |
The final code runs a version of Rocket, which is found at: | |
https://aochagavia.github.io/blog/rocket---a-rust-game-running-on-wasm/ | |
Rocket is a Rust program compiled to WebAssembly. I made no changes | |
to this code and used only the published program.wasm file found on | |
Aochagavia's site. | |
Here are some interesting links to projects involving WebAssembly and Python | |
1. The WebAssembly spec: https://webassembly.github.io/spec/ | |
2. Almar Klein's EuroPython 2018 talk. www.youtube.com/watch?v=u2kKxmb9BWs | |
Highly recommended. He does other neat stuff with Rocket. | |
3. pyodide. (Scientific Stack on Web Assembly) github.com/iodide-project/pyodide | |
4. Pure Python Compiler Infrastructure (PPCI). https://ppci.readthedocs.io | |
5. Wasmer (https://wasmer.io) | |
The "wadze" library that I used to decode Wasm is a creation of mine. The | |
official source code for that is available at https://github.com/dabeaz/wadze | |
Wadze is a real project. Feedback welcome! | |
*** IMPORTANT DISCLAIMER *** | |
This code was written/designed specifically for the purposes of giving | |
a talk and having something "work" in under an hour. It takes a number | |
of liberties with the Wasm spec, is incomplete, and is almost | |
certainly not anything you would ever want to use. That said: enjoy! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# machine.py | |
from numpy import ( | |
int32, int64, float32, float64 | |
) | |
import struct | |
_const = { | |
'i32.const' : int32, | |
'i64.const' : int64, | |
'f32.const' : float32, | |
'f64.const' : float64, | |
} | |
_binary = { | |
# 32-bit integer | |
'i32.add' : lambda x, y: int32(x + y), | |
'i32.sub' : lambda x, y: int32(x - y), | |
'i32.mul' : lambda x, y: int32(x * y), | |
'i32.div_s' : lambda x, y: int32(x // y), | |
'i32.div_u' : lambda x, y: int32(uint32(x) / uint32(y)), | |
'i32.and' : lambda x, y: int32(x & y), | |
'i32.or' : lambda x, y: int32(x | y), | |
'i32.xor' : lambda x, y: int32(x ^ y), | |
'i32.rem_s' : lambda x, y: int32(x % y), | |
'i32.rem_u' : lambda x, y: int32(uint32(x) // uint32(y)), | |
'i32.eq' : lambda x, y: int32(x == y), | |
'i32.ne' : lambda x, y: int32(x != y), | |
'i32.lt_s' : lambda x, y: int32(x < y), | |
'i32.le_s' : lambda x, y: int32(x <= y), | |
'i32.gt_s' : lambda x, y: int32(x > y), | |
'i32.ge_s' : lambda x, y: int32(x >= y), | |
'i32.lt_u' : lambda x, y: int32(uint32(x) < uint32(y)), | |
'i32.gt_u' : lambda x, y: int32(uint32(x) > uint32(y)), | |
'i32.le_u' : lambda x, y: int32(uint32(x) <= uint32(y)), | |
'i32.ge_u' : lambda x, y: int32(uint32(x) >= uint32(y)), | |
'i32.rotr' : lambda x, y: int32((x >> y) | ((x & ((2**y)-1)) << (32-y))), | |
'i32.rotl' : lambda x, y: int32((x << y) | ((x & ((2**y)-1)) >> (32-y))), | |
'i32.shr_u' : lambda x, y: int32(int(uint32(x)) >> int(y)), | |
'i32.shl' : lambda x, y: int32(x << y), | |
# 64-bit integer | |
'i64.add' : lambda x, y: int64(x + y), | |
'i64.sub' : lambda x, y: int64(x - y), | |
'i64.mul' : lambda x, y: int64(x * y), | |
'i64.div_s' : lambda x, y: int64(x // y), | |
'i64.div_u' : lambda x, y: int64(uint64(x) / uint64(y)), | |
'i64.and' : lambda x, y: int64(x & y), | |
'i64.or' : lambda x, y: int64(x | y), | |
'i64.xor' : lambda x, y: int64(x ^ y), | |
'i64.rem_s' : lambda x, y: int64(x % y), | |
'i64.rem_u' : lambda x, y: int64(uint64(x) // uint64(y)), | |
'i64.eq' : lambda x, y: int32(x == y), | |
'i64.ne' : lambda x, y: int32(x != y), | |
'i64.lt_s' : lambda x, y: int32(x < y), | |
'i64.le_s' : lambda x, y: int32(x <= y), | |
'i64.gt_s' : lambda x, y: int32(x > y), | |
'i64.ge_s' : lambda x, y: int32(x >= y), | |
'i64.lt_u' : lambda x, y: int32(uint64(x) < uint64(y)), | |
'i64.gt_u' : lambda x, y: int32(uint64(x) > uint64(y)), | |
'i64.le_u' : lambda x, y: int32(uint64(x) <= uint64(y)), | |
'i64.ge_u' : lambda x, y: int32(uint64(x) >= uint64(y)), | |
'i64.rotr' : lambda x, y: int64((x >> y) | ((x & ((2**y)-1)) << (64-y))), | |
'i64.rotl' : lambda x, y: int64((x << y) | ((x & ((2**y)-1)) >> (64-y))), | |
'i64.shr_u' : lambda x, y: int64(int(uint64(x)) >> int(y)), | |
'i64.shl' : lambda x, y: int64(x << y), | |
# -- 64 bit float | |
'f64.add' : lambda x, y: float64(x + y), | |
'f64.sub' : lambda x, y: float64(x - y), | |
'f64.mul' : lambda x, y: float64(x * y), | |
'f64.div' : lambda x, y: float64(x / y), | |
'f64.eq' : lambda x, y: int32(x == y), | |
'f64.ne' : lambda x, y: int32(x != y), | |
'f64.lt' : lambda x, y: int32(x < y), | |
'f64.gt' : lambda x, y: int32(x > y), | |
'f64.le' : lambda x, y: int32(x <= y), | |
'f64.ge' : lambda x, y: int32(x >= y), | |
} | |
import math | |
_unary = { | |
'i32.eqz' : lambda x: int32(x == 0), | |
'i32.clz' : lambda x: int32(32 - len(bin(uint32(x))[2:])), | |
'i32.ctz' : lambda x: int32(len(bin(uint32(x)).rsplit('1',1)[-1])), | |
'i32.wrap_i64' : lambda x: int32(uint64(x)), | |
'i64.extend_i32_u' : lambda x: int64(uint32(x)), | |
'f64.reinterpret_i64': lambda x: frombuffer(x.tobytes(), float64)[0], | |
'f64.sqrt': lambda x: float64(math.sqrt(x)), | |
'f64.convert_i32_u': lambda x: float64(uint32(x)), | |
} | |
from numpy import ( | |
int8, uint8, int16, uint16, uint32, uint64, frombuffer, | |
) | |
_load = { | |
'i32.load' : lambda raw: frombuffer(raw, int32)[0], | |
'i64.load' : lambda raw: frombuffer(raw, int64)[0], | |
'f64.load' : lambda raw: frombuffer(raw, float64)[0], | |
'i32.load8_s' : lambda raw: int32(frombuffer(raw, int8)[0]), | |
'i32.load8_u' : lambda raw: int32(frombuffer(raw, uint8)[0]), | |
'i32.load16_s' : lambda raw: int32(frombuffer(raw, int16)[0]), | |
'i32.load16_u' : lambda raw: int32(frombuffer(raw, uint16)[0]), | |
'i64.load8_s' : lambda raw: int64(frombuffer(raw, int8)[0]), | |
'i64.load8_u' : lambda raw: int64(frombuffer(raw, uint8)[0]), | |
'i64.load16_s' : lambda raw: int64(frombuffer(raw, int16)[0]), | |
'i64.load16_u' : lambda raw: int64(frombuffer(raw, uint16)[0]), | |
'i64.load32_s' : lambda raw: int64(frombuffer(raw, int32)[0]), | |
'i64.load32_u' : lambda raw: int64(frombuffer(raw, uint32)[0]), | |
} | |
_store = { | |
'i32.store' : lambda val: val.tobytes(), | |
'i64.store' : lambda val: val.tobytes(), | |
'f64.store' : lambda val: val.tobytes(), | |
'i32.store8' : lambda val: val.tobytes()[:1], | |
'i32.store16' : lambda val: val.tobytes()[:2], | |
'i64.store8' : lambda val: val.tobytes()[:1], | |
'i64.store16' : lambda val: val.tobytes()[:2], | |
'i64.store32' : lambda val: val.tobytes()[:4], | |
} | |
class Function: | |
def __init__(self, nparams, returns, code): | |
self.nparams = nparams | |
self.returns = returns | |
self.code = code | |
class ImportFunction: | |
def __init__(self, nparams, returns, call): | |
self.nparams = nparams | |
self.returns = returns | |
self.call = call | |
class Machine: | |
def __init__(self, functions, memsize=65536): | |
self.functions = functions # function table | |
self.items = [] | |
self.memory = bytearray(memsize) | |
def load(self, addr): | |
return struct.unpack('<d', self.memory[addr:addr+8])[0] | |
def store(self, addr, val): | |
self.memory[addr:addr+8] = struct.pack('<d', val) | |
def push(self, item): | |
assert type(item) in { int32, int64, float32, float64 } | |
self.items.append(item) | |
def pop(self): | |
return self.items.pop() | |
def call(self, func, *args): | |
locals = dict(enumerate(args)) # { 0: args[0], 1: args[1], 2: args[2] } | |
if isinstance(func, Function): | |
try: | |
self.execute(func.code, locals) | |
except Return: | |
pass | |
if func.returns: | |
return self.pop() | |
else: | |
return func.call(*args) # External (import function) | |
def execute(self, instructions, locals): | |
for op, *args in instructions: | |
# print(op, args, self.items) | |
if op in _const: | |
self.push(_const[op](args[0])) | |
elif op in _binary: | |
right = self.pop() | |
left = self.pop() | |
self.push(_binary[op](left, right)) | |
elif op in _unary: | |
self.push(_unary[op](self.pop())) | |
elif op in _load: | |
addr = self.pop() + args[1] # Offset | |
self.push(_load[op](self.memory[addr:addr+8])) | |
elif op in _store: | |
val = self.pop() | |
addr = self.pop() + args[1] | |
raw = _store[op](val) | |
self.memory[addr:addr+len(raw)] = raw | |
elif op == 'memory.size': | |
self.push(int32(len(self.memory)//65536)) | |
elif op == 'memory.grow': | |
npages = self.pop() | |
self.memory.extend(bytes(npages*65536)) | |
self.push(int32(len(self.memory)//65536)) | |
elif op == 'local.get': | |
self.push(locals[args[0]]) | |
elif op == 'local.set': | |
locals[args[0]] = self.pop() | |
elif op == 'local.tee': | |
locals[args[0]] = self.items[-1] | |
elif op == 'drop': | |
self.pop() | |
elif op == 'select': | |
c = self.pop() | |
v2 = self.pop() | |
v1 = self.pop() | |
self.push(v1 if c else v2) | |
elif op == 'call': | |
func = self.functions[args[0]] | |
fargs = reversed([ self.pop() for _ in range(func.nparams) ]) | |
result = self.call(func, *fargs) | |
if func.returns: | |
self.push(result) | |
elif op == 'br': | |
raise Break(args[0]) | |
elif op == 'br_if': | |
if self.pop(): | |
raise Break(args[0]) | |
elif op == 'br_table': # (br_tabel, [], default) | |
n = self.pop() | |
if n < len(args[0]): | |
raise Break(args[0][n]) | |
else: | |
raise Break(args[1]) | |
elif op == 'block': # ('block', type, [ instructions ]) | |
try: | |
self.execute(args[1], locals) | |
except Break as b: | |
if b.level > 0: | |
b.level -= 1 | |
raise | |
# if (test) { consequence } else {alternative } | |
# | |
# ('block', [ | |
# ('block', [ | |
# test | |
# ('br_if, 0), # Goto 0 | |
# alternative, | |
# ('br', 1), # Goto 1 | |
# ] | |
# ), # Label : 0 | |
# consequence, | |
# ] | |
# ) # Label 1: | |
elif op == 'loop': | |
while True: | |
try: | |
self.execute(args[1], locals) | |
break | |
except Break as b: | |
if b.level > 0: | |
b.level -= 1 | |
raise | |
# while (test) { body } | |
# ('block', [ | |
# ('loop', [ # Label 0 | |
# not test | |
# ('br_if', 1), # Goto 1: (break) | |
# body | |
# ('br', 0), # Goto 0: (continue) | |
# ] | |
# ) | |
# ] | |
# ) # label 1 | |
elif op == 'return': | |
raise Return() | |
else: | |
raise RuntimeError(f'Bad op {op}') | |
class Break(Exception): | |
def __init__(self, level): | |
self.level = level | |
class Return(Exception): | |
pass | |
def example(): | |
def py_display_player(x): | |
import time | |
print(' '*int(x) + '<O:>') | |
time.sleep(0.02) | |
display_player = ImportFunction(nparams=1, returns=None, call=py_display_player) | |
# def update_position(x, v, dt): | |
# return x + v*dt | |
# | |
update_position = Function(nparams=3, returns=True, code=[ | |
('local.get', 0), # x | |
('local.get', 1), # v | |
('local.get', 2), # dt | |
('mul',), | |
('add',) | |
]) | |
functions = [update_position, display_player] | |
# x = 2 | |
# v = 3 | |
# x = x + v*0.1 | |
x_addr = 22 | |
v_addr = 42 | |
m = Machine(functions) | |
m.store(x_addr, 2.0) | |
m.store(v_addr, 3.0) | |
# while x > 0 { | |
# x = update_position(x, v, 0.1) | |
# if x >= 70 { | |
# v = -v; | |
# } | |
# } | |
m.execute([ | |
('block', [ | |
('loop', [ | |
('const', x_addr), | |
('load',), | |
('call', 1), | |
('const', x_addr), | |
('load',), | |
('const', 0.0), | |
('le',), | |
('br_if', 1), | |
('const', x_addr), | |
('const', x_addr), | |
('load',), | |
('const', v_addr), | |
('load',), | |
('const', 0.1), | |
('call', 0), | |
('store',), | |
('block', [ | |
('const', x_addr), | |
('load',), | |
('const', 70.0), | |
('ge',), | |
('block', [ | |
('br_if', 0), | |
('br', 1), | |
] | |
), | |
('const', v_addr), | |
('const', 0.0), | |
('const', v_addr), | |
('load',), | |
('sub',), | |
('store',) | |
] | |
), | |
('br', 0), | |
] | |
) | |
] | |
) | |
], None) | |
print('Result:', m.load(x_addr)) | |
if __name__ == '__main__': | |
example() | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# rocket.py | |
import wadze # Web Assembly Decoder | |
module = wadze.parse_module(open('program.wasm', 'rb').read()) | |
# Build imported functions | |
import math | |
# These functions are imported by Wasm. Must be implemented in the host | |
# environment (Python). These are listed in the required order. | |
def imp_Math_atan(x): | |
return float64(math.atan(x)) | |
def imp_clear_screen(): | |
print('clear_screen') | |
def imp_cos(x): | |
return float64(math.cos(x)) | |
def imp_draw_bullet(x, y): | |
print('draw_bullet', x, y) | |
def imp_draw_enemy(x, y): | |
print('draw_enemy', x, y) | |
def imp_draw_particle(x, y, z): | |
print('draw_particle', x, y, z) | |
def imp_draw_player(x, y, z): | |
print('draw_player', x, y, z) | |
def imp_draw_score(s): | |
print('draw_score', s) | |
def imp_sin(x): | |
return float64(math.sin(x)) | |
import pygame | |
pygame.init() | |
size = width, height = (800, 600) | |
screen = pygame.display.set_mode(size) | |
# font = pygame.font.SysFont("helvetica", 36) | |
def imp_clear_screen(): | |
screen.fill((0,0,0)) | |
def imp_draw_bullet(x, y): | |
pygame.draw.circle(screen, (255, 255, 255), (int(x), int(y)), 2) | |
def imp_draw_enemy(x, y): | |
pygame.draw.circle(screen, (255,0,255), (int(x), int(y)), 15, 2) | |
def imp_draw_particle(x, y, z): | |
pygame.draw.circle(screen, (255,255,0), (int(x),int(y)), abs(int(z))) | |
def imp_draw_player(x, y, z): | |
pygame.draw.circle(screen, (0,255,255), (int(x), int(y)), 10, 2) | |
def imp_draw_score(s): | |
print('draw_score', s) | |
# text = font.render(f'Score: {int(s)}', True, (200,200, 0)) | |
# screen.blit(text, (5, 5)) | |
import machine | |
# Declare as imported functions for our "machine" | |
imported_functions = [ | |
machine.ImportFunction(1, 1, imp_Math_atan), | |
machine.ImportFunction(0, 0, imp_clear_screen), | |
machine.ImportFunction(1, 1, imp_cos), | |
machine.ImportFunction(2, 0, imp_draw_bullet), | |
machine.ImportFunction(2, 0, imp_draw_enemy), | |
machine.ImportFunction(3, 0, imp_draw_particle), | |
machine.ImportFunction(3, 0, imp_draw_player), | |
machine.ImportFunction(1, 0, imp_draw_score), | |
machine.ImportFunction(1, 1, imp_sin), | |
] | |
# Declare "defined" functions | |
defined_functions = [ ] | |
for typeidx, code in zip(module['func'], module['code']): | |
functype = module['type'][typeidx] # Signature | |
func = machine.Function(nparams = len(functype.params), | |
returns = bool(functype.returns), | |
code = wadze.parse_code(code).instructions) | |
defined_functions.append(func) | |
functions = imported_functions + defined_functions | |
# Declare "exported" functions | |
exports = { exp.name: functions[exp.ref] for exp in module['export'] | |
if isinstance(exp, wadze.ExportFunction) } | |
m = machine.Machine(functions, 20*65536) # Hack on memory | |
# Initialize memory | |
for data in module['data']: | |
m.execute(data.offset, None) | |
offset = m.pop() | |
m.memory[offset:offset+len(data.values)] = data.values | |
from machine import float64, int32 | |
# Call something | |
width = float64(800.0) | |
height = float64(600.) | |
m.call(exports['resize'], width, height) # Prayer | |
# Game loop | |
import time | |
last = time.time() | |
while True: | |
for event in pygame.event.get(): | |
if event.type == pygame.QUIT: | |
raise SystemExit() | |
elif event.type == pygame.KEYUP: | |
if event.key == 32: | |
m.call(exports['toggle_shoot'], int32(0)) | |
elif event.key == 275: | |
m.call(exports['toggle_turn_right'], int32(0)) | |
elif event.key == 276: | |
m.call(exports['toggle_turn_left'], int32(0)) | |
elif event.key == 273: | |
m.call(exports['toggle_boost'], int32(0)) | |
elif event.type == pygame.KEYDOWN: | |
if event.key == 32: | |
m.call(exports['toggle_shoot'], int32(1)) | |
elif event.key == 275: | |
m.call(exports['toggle_turn_right'], int32(1)) | |
elif event.key == 276: | |
m.call(exports['toggle_turn_left'], int32(1)) | |
elif event.key == 273: | |
m.call(exports['toggle_boost'], int32(1)) | |
now = time.time() | |
dt = now - last | |
last = now | |
m.call(exports['update'], float64(dt)) | |
m.call(exports['draw']) | |
pygame.display.flip() | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# wadze.py | |
# | |
# Web Assembly Decoder - Zero Extras | |
# | |
# Copyright (C) 2019 | |
# David M. Beazley (https://www.dabeaz.com) | |
# All Rights Reserved | |
# | |
# Redistribution and use in source and binary forms, with or without | |
# modification, are permitted provided that the following conditions are | |
# met: | |
# * Redistributions of source code must retain the above copyright notice, | |
# this list of conditions and the following disclaimer. | |
# * Redistributions in binary form must reproduce the above copyright notice, | |
# this list of conditions and the following disclaimer in the documentation | |
# and/or other materials provided with the distribution. | |
# * Neither the name of the David Beazley or Dabeaz LLC may be used to | |
# endorse or promote products derived from this software without | |
# specific prior written permission. | |
# | |
# This software is provided "as is." The risks of using it are yours. | |
from collections import namedtuple | |
from itertools import islice | |
import struct | |
_typemap = { | |
0x7f : 'i32', | |
0x7e : 'i64', | |
0x7d : 'f32', | |
0x7c : 'f64', | |
0x70 : 'anyfunc', | |
0x60 : 'func', | |
0x40 : None, | |
} | |
def parse_unsigned(stream): | |
result = shift = 0 | |
while True: | |
b = next(stream) | |
result |= (b & 0x7f) << shift | |
shift += 7 | |
if not (b & 0x80): | |
break | |
return result | |
def parse_signed(stream): | |
result = shift = 0 | |
while True: | |
b = next(stream) | |
result |= (b & 0x7f) << shift | |
shift += 7 | |
if not (b & 0x80): | |
break | |
if b & 0x40: | |
result |= (~0 << shift) | |
return result | |
def parse_float32(stream): | |
return struct.unpack('<f', bytes(islice(stream, 4)))[0] | |
def parse_float64(stream): | |
return struct.unpack('<d', bytes(islice(stream, 8)))[0] | |
def parse_vector(stream, func): | |
return [ func(stream) for _ in range(parse_unsigned(stream)) ] | |
def parse_string(stream): | |
return bytes(parse_vector(stream, next)).decode('utf-8') | |
FunctionType = namedtuple('FunctionType', ['params', 'returns']) | |
def parse_functype(stream): | |
sigtype = next(stream) | |
params = [ _typemap[t] for t in parse_vector(stream, next) ] | |
returns = [ _typemap[t] for t in parse_vector(stream, next) ] | |
return FunctionType(params, returns) | |
Limits = namedtuple('Limits', ['min', 'max']) | |
def parse_limits(stream): | |
return Limits(parse_unsigned(stream), parse_unsigned(stream)) if next(stream) else (parse_unsigned(stream), None) | |
TableType = namedtuple('TableType', ['elemtype', 'limits']) | |
def parse_tabletype(stream): | |
return TableType(_typemap[next(stream)], parse_limits(stream)) | |
GlobalType = namedtuple('GlobalType', ['type', 'mut']) | |
def parse_globaltype(stream): | |
return GlobalType(_typemap[next(stream)], next(stream)) | |
ImportFunction = namedtuple('ImportFunction', ['module','name','typeidx']) | |
ImportTable = namedtuple('ImportTable', ['module','name', 'tabletype']) | |
ImportMemory = namedtuple('ImportMemory', ['module', 'name', 'limits']) | |
ImportGlobal = namedtuple('ImportGlobal', ['module', 'name', 'globaltype']) | |
_imports = { | |
0 : (ImportFunction, parse_unsigned), | |
1 : (ImportTable, parse_tabletype), | |
2 : (ImportMemory, parse_limits), | |
3 : (ImportGlobal, parse_globaltype), | |
} | |
def parse_import(stream): | |
module = parse_string(stream) | |
name = parse_string(stream) | |
cls, func = _imports[next(stream)] | |
return cls(module, name, func(stream)) | |
Global = namedtuple('Global', ['globaltype', 'expr']) | |
def parse_global(stream): | |
return Global(parse_globaltype(stream), parse_instructions(stream)) | |
ExportFunction = namedtuple('ExportFunction', ['name', 'ref']) | |
ExportTable = namedtuple('ExportTable', ['name', 'ref']) | |
ExportMemory = namedtuple('ExportMemory', ['name', 'ref']) | |
ExportGlobal = namedtuple('ExportGlobal', ['name', 'ref']) | |
_exports = { 0: ExportFunction, 1: ExportTable, 2: ExportMemory, 3: ExportGlobal } | |
def parse_export(stream): | |
name = parse_string(stream) | |
return _exports[next(stream)](name, parse_unsigned(stream)) | |
Element = namedtuple('Element', ['tableidx', 'offset', 'values']) | |
def parse_element(stream): | |
return Element(parse_unsigned(stream), parse_instructions(stream), parse_vector(stream, parse_unsigned)) | |
def parse_locals(stream): | |
return parse_unsigned(stream) * (_typemap[next(stream)],) | |
Code = namedtuple('Code', ['locals', 'instructions']) | |
def parse_rawcode(stream): | |
return bytes(islice(stream, parse_unsigned(stream))) | |
def parse_code(raw): | |
if isinstance(raw, bytes): | |
code = iter(raw) | |
locals = [ ] | |
for loc in parse_vector(code, parse_locals): | |
locals.extend(loc) | |
instructions = parse_instructions(code) | |
return Code(locals, instructions) | |
Data = namedtuple('Data', ['memidx', 'offset', 'values']) | |
def parse_data(stream): | |
return Data(parse_unsigned(stream), parse_instructions(stream), parse_vector(stream, next)) | |
def parse_instructions(stream): | |
instructions = [ ] | |
while True: | |
op = next(stream) | |
if op == 0x0b: | |
break | |
name, *funcs = _opcodes[op] | |
instructions.append((name, *(func(stream) for func in funcs))) | |
return instructions | |
def _split_else(instructions): | |
if ('else',) in instructions: | |
index = instructions.index(('else',)) | |
return (instructions[:index], instructions[index+1:]) | |
else: | |
return (instructions, []) | |
_opcodes = { | |
0x00 : ('unreachable', ), | |
0x01 : ('nop', ), | |
0x02 : ('block', lambda s: _typemap[next(s)], parse_instructions), | |
0x03 : ('loop', lambda s: _typemap[next(s)], parse_instructions), | |
0x04 : ('if', lambda s: _typemap[next(s)], lambda s: _split_else(parse_instructions(s))), | |
0x05 : ('else',), | |
0x0c : ('br', parse_unsigned), | |
0x0d : ('br_if', parse_unsigned), | |
0x0e : ('br_table', lambda s: parse_vector(s, parse_unsigned), parse_unsigned), | |
0x0f : ('return', ), | |
0x10 : ('call', parse_unsigned), | |
0x11 : ('call_indirect', parse_unsigned, next), | |
0x1a : ('drop', ), | |
0x1b : ('select', ), | |
0x20 : ('local.get', parse_unsigned), | |
0x21 : ('local.set', parse_unsigned), | |
0x22 : ('local.tee', parse_unsigned), | |
0x23 : ('global.get', parse_unsigned), | |
0x24 : ('global.set', parse_unsigned), | |
0x28 : ('i32.load', parse_signed, parse_signed), | |
0x29 : ('i64.load', parse_signed, parse_signed), | |
0x2a : ('f32.load', parse_signed, parse_signed), | |
0x2b : ('f64.load', parse_signed, parse_signed), | |
0x2c : ('i32.load8_s', parse_signed, parse_signed), | |
0x2d : ('i32.load8_u', parse_signed, parse_signed), | |
0x2e : ('i32.load16_s', parse_signed, parse_signed), | |
0x2f : ('i32.load16_u', parse_signed, parse_signed), | |
0x30 : ('i64.load8_s', parse_signed, parse_signed), | |
0x31 : ('i64.load8_u', parse_signed, parse_signed), | |
0x32 : ('i64.load16_s', parse_signed, parse_signed), | |
0x33 : ('i64.load16_u', parse_signed, parse_signed), | |
0x34 : ('i64.load32_s', parse_signed, parse_signed), | |
0x35 : ('i64.load32_u', parse_signed, parse_signed), | |
0x36 : ('i32.store', parse_signed, parse_signed), | |
0x37 : ('i64.store', parse_signed, parse_signed), | |
0x38 : ('f32.store', parse_signed, parse_signed), | |
0x39 : ('f64.store', parse_signed, parse_signed), | |
0x3a : ('i32.store8', parse_signed, parse_signed), | |
0x3b : ('i32.store16', parse_signed, parse_signed), | |
0x3c : ('i64.store8', parse_signed, parse_signed), | |
0x3d : ('i64.store16', parse_signed, parse_signed), | |
0x3e : ('i64.store32', parse_signed, parse_signed), | |
0x3f : ('memory.size', next), | |
0x40 : ('memory.grow', next), | |
0x41 : ('i32.const', parse_signed), | |
0x42 : ('i64.const', parse_signed), | |
0x43 : ('f32.const', parse_float32), | |
0x44 : ('f64.const', parse_float64), | |
0x45 : ('i32.eqz', ), | |
0x46 : ('i32.eq', ), | |
0x47 : ('i32.ne', ), | |
0x48 : ('i32.lt_s', ), | |
0x49 : ('i32.lt_u', ), | |
0x4a : ('i32.gt_s', ), | |
0x4b : ('i32.gt_u', ), | |
0x4c : ('i32.le_s', ), | |
0x4d : ('i32.le_u', ), | |
0x4e : ('i32.ge_s', ), | |
0x4f : ('i32.ge_u', ), | |
0x50 : ('i64.eqz', ), | |
0x51 : ('i64.eq', ), | |
0x52 : ('i64.ne', ), | |
0x53 : ('i64.lt_s', ), | |
0x54 : ('i64.lt_u', ), | |
0x55 : ('i64.gt_s', ), | |
0x56 : ('i64.gt_u', ), | |
0x57 : ('i64.le_s', ), | |
0x58 : ('i64.le_u', ), | |
0x59 : ('i64.ge_s', ), | |
0x5a : ('i64.ge_u', ), | |
0x5b : ('f32.eq', ), | |
0x5c : ('f32.ne', ), | |
0x5d : ('f32.lt', ), | |
0x5e : ('f32.gt', ), | |
0x5f : ('f32.le', ), | |
0x60 : ('f32.ge', ), | |
0x61 : ('f64.eq', ), | |
0x62 : ('f64.ne', ), | |
0x63 : ('f64.lt', ), | |
0x64 : ('f64.gt', ), | |
0x65 : ('f64.le', ), | |
0x66 : ('f64.ge', ), | |
0x67 : ('i32.clz', ), | |
0x68 : ('i32.ctz', ), | |
0x69 : ('i32.popcnt', ), | |
0x6a : ('i32.add', ), | |
0x6b : ('i32.sub', ), | |
0x6c : ('i32.mul', ), | |
0x6d : ('i32.div_s', ), | |
0x6e : ('i32.div_u', ), | |
0x6f : ('i32.rem_s', ), | |
0x70 : ('i32.rem_u', ), | |
0x71 : ('i32.and', ), | |
0x72 : ('i32.or', ), | |
0x73 : ('i32.xor', ), | |
0x74 : ('i32.shl', ), | |
0x75 : ('i32.shr_s', ), | |
0x76 : ('i32.shr_u', ), | |
0x77 : ('i32.rotl', ), | |
0x78 : ('i32.rotr', ), | |
0x79 : ('i64.clz', ), | |
0x7a : ('i64.ctz', ), | |
0x7b : ('i64.popcnt', ), | |
0x7c : ('i64.add', ), | |
0x7d : ('i64.sub', ), | |
0x7e : ('i64.mul', ), | |
0x7f : ('i64.div_s', ), | |
0x80 : ('i64.div_u', ), | |
0x81 : ('i64.rem_s', ), | |
0x82 : ('i64.rem_u', ), | |
0x83 : ('i64.and', ), | |
0x84 : ('i64.or', ), | |
0x85 : ('i64.xor', ), | |
0x86 : ('i64.shl', ), | |
0x87 : ('i64.shr_s', ), | |
0x88 : ('i64.shr_u', ), | |
0x89 : ('i64.rotl', ), | |
0x8a : ('i64.rotr', ), | |
0x8b : ('f32.abs', ), | |
0x8c : ('f32.neg', ), | |
0x8d : ('f32.ceil', ), | |
0x8e : ('f32.floor', ), | |
0x8f : ('f32.trunc', ), | |
0x90 : ('f32.nearest', ), | |
0x91 : ('f32.sqrt', ), | |
0x92 : ('f32.add', ), | |
0x93 : ('f32.sub', ), | |
0x94 : ('f32.mul', ), | |
0x95 : ('f32.div', ), | |
0x96 : ('f32.min', ), | |
0x97 : ('f32.max', ), | |
0x98 : ('f32.copysign', ), | |
0x99 : ('f64.abs', ), | |
0x9a : ('f64.neg', ), | |
0x9b : ('f64.ceil', ), | |
0x9c : ('f64.floor', ), | |
0x9d : ('f64.trunc', ), | |
0x9e : ('f64.nearest', ), | |
0x9f : ('f64.sqrt', ), | |
0xa0 : ('f64.add', ), | |
0xa1 : ('f64.sub', ), | |
0xa2 : ('f64.mul', ), | |
0xa3 : ('f64.div', ), | |
0xa4 : ('f64.min', ), | |
0xa5 : ('f64.max', ), | |
0xa6 : ('f64.copysign', ), | |
0xa7 : ('i32.wrap_i64', ), | |
0xa8 : ('i32.trunc_f32_s', ), | |
0xa9 : ('i32.trunc_f32_u', ), | |
0xaa : ('i32.trunc_f64_s', ), | |
0xab : ('i32.trunc_f64_u', ), | |
0xac : ('i64.extend_i32_s', ), | |
0xad : ('i64.extend_i32_u', ), | |
0xae : ('i64.trunc_f32_s', ), | |
0xaf : ('i64.trunc_f32_u', ), | |
0xb0 : ('i64.trunc_f64_s', ), | |
0xb1 : ('i64.trunc_f64_u', ), | |
0xb2 : ('f32.convert_i32_s', ), | |
0xb3 : ('f32.convert_i32_u', ), | |
0xb4 : ('f32.convert_i64_s', ), | |
0xb5 : ('f32.convert_i64_u', ), | |
0xb6 : ('f32.demote_f64', ), | |
0xb7 : ('f64.convert_i32_s', ), | |
0xb8 : ('f64.convert_i32_u', ), | |
0xb9 : ('f64.convert_i64_s', ), | |
0xba : ('f64.convert_i64_u', ), | |
0xbb : ('f64.promote_f32', ), | |
0xbc : ('i32.reinterpret_f32', ), | |
0xbd : ('i64.reinterpret_f64', ), | |
0xbe : ('f32.reinterpret_i32', ), | |
0xbf : ('f64.reinterpret_i64', ), | |
} | |
_sections = { | |
1 : ('type', lambda s: parse_vector(s, parse_functype)), | |
2 : ('import', lambda s: parse_vector(s, parse_import)), | |
3 : ('func', lambda s: parse_vector(s, parse_unsigned)), | |
4 : ('table', lambda s: parse_vector(s, parse_tabletype)), | |
5 : ('memory', lambda s: parse_vector(s, parse_limits)), | |
6 : ('global', lambda s: parse_vector(s, parse_global)), | |
7 : ('export', lambda s: parse_vector(s, parse_export)), | |
8 : ('start', parse_unsigned), | |
9 : ('element', lambda s: parse_vector(s, parse_element)), | |
10 : ('code', lambda s: parse_vector(s, parse_rawcode)), | |
11 : ('data', lambda s: parse_vector(s, parse_data)), | |
} | |
def parse_section(stream): | |
sectnum = next(stream) | |
size = parse_unsigned(stream) | |
if sectnum in _sections: | |
sectname, parse_func = _sections[sectnum] | |
return (sectname, parse_func(stream)) | |
else: | |
return (sectnum, bytes(islice(stream, size))) | |
def parse_module(stream): | |
''' | |
Parse binary .wasm format data into a dictionary representing the | |
different sections of a Wasm module. stream is either an iterator | |
producing bytes or a byte string containing the raw data from a | |
.wasm file. | |
''' | |
if isinstance(stream, bytes): | |
stream = iter(stream) | |
header = bytes(islice(stream, 8)) | |
if header[:4] != b'\x00asm': | |
raise ValueError('Expected .wasm') | |
sections = { } | |
while True: | |
try: | |
sectname, items = parse_section(stream) | |
sections[sectname] = items | |
except StopIteration: | |
break | |
return sections | |
# Example use | |
if __name__ == '__main__': | |
module = parse_module(open('input.wasm','rb').read()) | |
# If you want to parse instruction code, also include this | |
module['code'] = [ parse_code(c) for c in module['code']] | |
wadze and rocket have licenses. Can we consider machine.py
to be in the public domain?
I'm developing a super-slim educational programming language that I want to remain in the public domain, and your talk was super inspiring for my approach to compiletime execution of const/macro functions.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is a very concise, but imminently readable, description. Thank you. Does this exist as a git repo somewhere so we can fix the bugs?