micropython-native.md

Limitations and pitfalls of @micropython.native

Micropython is fully compatible with Python 3 syntax... except when it isn't.

In this article, we explore one of the "dark corners" of micropython: limitations imposed by @micropython.native decorator.

Documentation on The Native code emitter is brief and a bit vague:

• Context managers are not supported (the with statement).

• Generators are not supported.

• If raise is used an argument must be supplied.

So, what exactly works and what exactly fails? Are there any differences in how Micropython v1.9.3 behaves on Unix and on ESP8266? Let's find out.

Generators don't work with @micropython.native

This is the easiest one: yield and @micropython.native just don't mix.

>>> @micropython.native
... def native_yield():
...         yield 1
...
NotImplementedError: native yield

It's the same error on Unix and on ESP8266.

But what if we call another generator from a native function?

>>> def generate():
...     yield 1
...
>>> @micropython.native
... def native_call_yield():
...     for x in generate():
...         return x
...
>>> native_call_yield()
1

Looks like Micropython has no problem with that.

except and raise and @micropython.native

From the documentation, it may seem that raise without an argument will just fail at compile time with NotImplementedError, the same way as yield. But actual behavior is more complex.

Explicit exception name works fine, of course:

@micropython.native
def native_ex_raise():
    try:
        ...
    except SomeException as ex:
        # do something...
        raise ex

To much surprise, implicit except/raise also seems to work, at least in our minimal example:

@micropython.native
def native_no_ex_no_raise():
    try:
        ...
    except SomeException:
        # do something...
        raise

This is a popular Python idiom, for two reasons:

1. In Python 2, raise without argument preserves the original stack trace (raise ex doesn't).

2. It's clear and concise.

But should we use it in Micropython? Probably no, because pfalcon himself said that "except" without "as" is strongly discouraged.

Now, let's try to mix except ... as ... and raise. It's perfectly legal in "normal" Python, but in Micropython with Native code emitter it fails; moreover, it is inconsistent across architectures.

Unix:

>>> @micropython.native
... def native_ex_no_raise():
...     try:
...         raise SomeException('BOOM!')
...     except SomeException as ex:
...         print('catch: native_ex_no_raise')
...         raise
...
>>> native_ex_no_raise()
catch: native_ex_no_raise
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: exceptions must derive from BaseException

ESP8266:

>>> @micropython.native
... def native_ex_no_raise():
...     try:
...         raise SomeException('BOOM!')
...     except SomeException as ex:
...         print('catch: native_ex_no_raise')
...         raise
...
MemoryError: memory allocation failed, allocating 368 bytes for native code

@micropython.native + custom exceptions = weirdness

Working with exceptions in Native functions may be generally a bad idea.

>>> with open('weird.py', 'w') as f:
...     f.write('import micropython\n')
...     f.write('class SomeException(Exception):\n')
...     f.write('  pass\n')
...     f.write('@micropython.native\n')
...     f.write('def raise_some():\n')
...     f.write('  raise SomeException\n')
...     f.write('raise_some()\n')
...
>>> import weird
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "weird.py", line 7, in <module>
SomeException:

Everythings is fine so far.

>>> weird.raise_some()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'weird' is not defined

OK, module was not imported because exception was raised. But, surprisingly, it imports successfully on the second try (in CPython, it would fail every time):

>>> import weird
>>> weird.SomeException
<class 'SomeException'>
>>> weird.raise_some
<function>

Can we call the function now?

>>> weird.raise_some()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'SomeException' is not defined

At first glance, it looks like a programmer's error, but it's not; and the same code without @micropython.native works fine. Apparently, the Native code emitter has some additional undocumented restrictions. This deserves further investigation.

@micropython.native and context managers

One more pitfall. What works on Unix...

>>> @micropython.native
... def native_with():
...     with open('a.txt', 'w') as file:
...         file.write('Hello world')
...
>>> native_with()
>>> print(open('a.txt').read())
Hello world

...fails on ESP8266:

>>> @micropython.native
... def native_with():
...     with open('a.txt', 'w') as file:
...         file.write('Hello world')
...
MemoryError: memory allocation failed, allocating 452 bytes for native code

Maybe open is special? Nope, if you write a custom context manager, behavior is the same: works on Unix, fails on ESP8266.

Conclusion

1. In some cases, Micropython behaves differently on Unix and on ESP8266.

2. Use except SomeException as ex: and raise ex. Don't use except SomeException: and raise without an argument.

3. Be careful with @micropython.native.