Skip to content

Instantly share code, notes, and snippets.

@tito
Last active May 10, 2018 10:04
Show Gist options
  • Save tito/9414743 to your computer and use it in GitHub Desktop.
Save tito/9414743 to your computer and use it in GitHub Desktop.
Cython optimization ideas (goal: reduce size for mobile)

Cython optimization ideas

Theses ideas is not good or bad, but are just experimentation to see the cost of a feature on the final binary size. All the tests came from the original execution of cython on kivy/graphics/texture.pyx. 40927 bytes.

A fresh compile using the default options give:

original: 899262
stripped: 187200

Idea 1: externalize Cython as a library

This is just a theory where all the cython functions would be declared and compiled aside, and shipped into the final binary. Instead of having N times the functions in the final binary, we would have it only once. Kivy with the default options compile 22 cython extensions. Cython is also used in other python-for-android recipes as android, ffmpeg, pyjnius, twisted, audiostream, lxml, cymunk, etc.

Here, the test consist to remove all the static functions from the generated .c, and rename the static proto to extern:

original: 770481
stripped: 170848

The gain is:

original: 128781
stripped: 16352

With lot of approx, the saving on the Kivy library itself would be:

stripped: 16352 * 22 = 359744

Idea 2: remove traceback informations and generation

I tested the idea 2 upon the previous .c, with the size:

original: 770481
stripped: 170848

First try, remove filename / lineno / clineo when a call failed:

:%sno/__pyx_filename = __pyx_f[\d\*]; __pyx_lineno = \d\*; __pyx_clineno = \(__LINE__\|\d\*\);//g
-> 856 lines removed

The size result for the first try is:

original: 662671
stripped: 150368

Next try, remove traceback generation:

:%sno/__Pyx_AddTraceback(\.\*);//g
-> 51 lines removed

The size of the second try is:

original: 652348
stripped: 150368

The gain is:

original: 118133
stripped: 20480

Idea 3: remove all the docstrings

This is not a new idea, as it was already used by default on python-for-android and kivy-ios... before. This is not the case anymore, as Cython was having an issue using -D. This should be restored on both toolchain by default now.

The original .c with docstrings is:

original: 899262
stripped: 187200

After removing the docstrings (cython -D)

original: 886517
stripped: 176544

The gain is:

original: 12745
stripped: 10656

Cumulative gain for texture.so

Original stripped size was187200
- idea 1 (externalize cython lib)- 16352
- idea 2 (remove traceback info)- 20480
- idea 3 (remove docstrings)- 10656
Final stripped size139712

⇒ ~25% file size gain, without changing the gcc compilation

Cumulative gain for texture.so with -Os

Original stripped size was187200
-Os stripped size is127408
- idea 1 (externalize cython lib)- 16352
- idea 2 (remove traceback info)- 20480
- idea 3 (remove docstrings)- 10656
Final stripped size79920 *
  • The ideas have been tested on -O2, not -Os. The impact would be smaller as well.

⇒ ~42% file size gain, using all the ideas + -Os

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment