Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Thoughts on Package Management for CircuitPython

Package Management for CircuitPython

Nothing formal written up on this yet, but I think we're seeing the download the zip file approach start to become brittle. Already there are two different bundles. My initial thought is why not have a cpip (or modify circup) and use PyPi. One thing I like about this is that it teaches new programmers things they need to learn if they also do Python programming. I've not done any real research into the technical parts of this yet, but in an ideal world, a fork of pip or some modular components of pip could be used to drive the core logic of version checking and install and the CP specific parts that deal with the board would be the additional functionality.

Copy link

askpatrickw commented Jan 14, 2021

I should add... its not that the .PY versions of the packages won't work, its that they're not compressed which helps with storage space AND pip installs the CPython dependencies not needed on the CircuitPython Board.

You can also use --path on pip list and pip freeze. It does not understand the .mpy packages of course. There is also a --platform option which I don't understand yet.

Note: You can see in these --path examples the "extra" CPython packages which were installed on my FeatherS2.

pip list --path /Volumes/CIRCUITPY/lib
Package                          Version
-------------------------------- -------
Adafruit-Blinka                  5.10.0
adafruit-circuitpython-busdevice 5.0.3
adafruit-circuitpython-pm25      2.1.0
Adafruit-PlatformDetect          2.25.0
Adafruit-PureIO                  1.1.8
pyftdi                           0.52.9
pyserial                         3.5
pyusb                            1.1.0
pip freeze --path /Volumes/CIRCUITPY/lib

Copy link

dglaude commented Apr 5, 2021

Wait, you are using pip to install CP library on a CP board mounted on the Linux host?
That is ... impressive? Mind blowing?

For the *.py vs *.mpy then maybe it could be downloaded to a temporary location and then mpy-cross to have the mpy and then only copy to the device.

Not sure what kind of wrapper that would require, but it might be a good trick.

Copy link

askpatrickw commented Apr 7, 2021

@dglaude, I'm not, but it looks like you could. ;-)

That's a great idea as an interim step.
I've not used MPY-Cross. When I look at it, it sounds like it's target OS specific. Am I misunderstanding that?

I notice this issue in CP Build Tools... maybe the mpy-cross referenced there could be used.

Copy link

dglaude commented Apr 7, 2021

I think that while MPY-Cross exist on multiple platform, it always generate the same output MPY file for the same input PY file.
It is "just" cross compiling, or doing the parsing from Python file to file that can be executed in the CircuitPython virtual machine.

Yes the mpy-cross for Windows/Mac/??? are build and available from there, so there is no need to compile MPY-Cross yourself.

Copy link

askpatrickw commented Jun 27, 2021

Could pip --implementation have mpy added?

Only use wheels compatible with Python implementation , e.g. 'pp', 'jy', 'cp', or
'ip'. If not specified, then the current interpreter implementation is used. Use 'py' to force
implementation-agnostic wheels.

Copy link

askpatrickw commented Jun 27, 2021

Or a CP PyPi...

Package Index Options:
-i, --index-url Base URL of the Python Package Index (default This should point to a
repository compliant with PEP 503 (the simple repository API) or a local directory laid out in the
same format.
--extra-index-url Extra URLs of package indexes to use in addition to --index-url. Should follow the same rules as
--no-index Ignore package index (only looking at --find-links URLs instead).
-f, --find-links If a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel
(.whl) files. If a local path or file:// URL that's a directory, then look for archives in the
directory listing. Links to VCS project URLs are not supported.

Copy link

aivarannamaa commented Jun 28, 2021

@askpatrickw, Here is my investigation concerning --implementation: aivarannamaa/minipip#6

You may also find aivarannamaa/minipip#5 and interesting

Copy link

askpatrickw commented Aug 11, 2021

Copy link

Thynix commented Jan 11, 2022

I haven't seen mention of it, so I'll offer that Mu Editor support for whatever the package management ends up looking like would be helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment