This link is a summary of the work I did for the Heat library as part of Google Summer of Code 2022.
- Title: Python array API standard compatibility
- Organization: Forschungszentrum Jülich
- Repository: Heat
- Mentor: Michael Tarnawa
- My GitHub profile: neosunhan
The primary goal of this project was to ensure conformance of the Heat library to the Python array API standard.
Heat is a distributed tensor framework for high performance data analytics. The Python array API standard aims to standardize multidimensional array libraries written in Python. To this end, it defines a set of functionalities that are supported by all libraries adhering to this standard.
In accordance with the array API standard, I created a new namespace to contain the functionalities specified in the API specification. This namespace is then accessed through a method on the array object.
xp = x.__array_namespace__()I then added the required functions into this new namespace.
To help ensure conformance to the specification, the accompanying test suite will be added to the CI workflow.
The array API standard provides detailed specification on the functionalities contained within, as well as comprehensive implementation details. Part of my work on this project included the testing of the functions within the API to identify deviations from the specification, and subsequent fixes to ensure conformance to the array API standard.
- Add
copykwarg toasarray - Precision loss in float arrays
- Non-negative sample size for
linspace/logspace minvalue of data types iniinfo/finfokeepdiminany/all- Rename
keepdimkwarg prod/sumwith empty arrays- Extra dimension on 0-dimensional arrays
Although most of the functionalities in the array API standard have been implemented, there are still some areas for improvement to bring the API closer to full compliance. The following suggestions would be a good starting point for future work:
- Implementation of DLPack support for the Heat Library. This will provide a data-interchange mechanism to convert arrays between different libraries.
- The Linear Algebra extension in the array API standard specifices a set of optional linear algebra functionalities that can be included in the array API.
- Several functions in the array API specification are currently being worked on (
tensordot,argsort,broadcast_arrays,broadcast_to). - An indexing and search overhaul in the Heat library is currently in progress that will fix many of the observed deviations from the specification.
I would like to thank Michael Tarnawa, Claudia Comito, and the rest of the Heat community for their guidance and support during this project.
Several of the PRs are still under review, and I will most likely need to make additional changes during the merging process. As mentioned above, there is still work to done to make the API of Heat standard-compliant, and I hope to contribute more in the future to this end.