For the summer of 2022 I participated in Google Summer of Code 2022 at MIT App Inventor. This gist summarises all the work I did during the program.
MIT App Inventor is a web based programming environment with over a million unique monthly users. It was originally provided by Google but now is maintained by MIT. It uses an intuitive GUI that provides drag and drop support to drop visual objects that let users create mobile applications. For programming logic it uses Blockly, a visual programming language.
For Google Summer of Code 2022, my work with MIT App Inventor was to create an extension component, so that the models trained in Teachable Machine can also be used in app inventor.
Teachable Machine is a web-based tool that makes creating machine learning models fast, easy, and accessible to everyone. No prior coding experience is required to create and train these models. For now, it provides three kinds of model -
- Image Classification
- Audio Classification
- Pose
For ML models, App Inventor also provides all three types of models - Image Classification , Audio Classification and Pose. For training of these model, App Inventor had provided links where users can train their own models, and can use them in App Inventor.
But the models trained in Teachable Machine cannot be used in App Inventor. So this summer, I worked on making a extension, where the user can also use the models trained in Teachable Machine. One more Feature that was added, that Pose models in App Inventor were already trained so users had to use the already trained model, they cannot use the pose model trained by them.
I worked on creating an extension, where users can use image classification models trained in Teachable Machine.
Below is the link to the PR, that contains the work I had done this summer. Teachable Machine - Image Classification
I am going to work with the MIT App Inventor for a little longer for the below features to be implemented. There is still some features that are yet to be implemented:-
- Teachable Machine - Audio Classification
- Teachable Machine - Pose
There are some features that can be implemented after the above work gets implemented:-
- Giving users an option whether they want to insert the model link or model file exported from Teachable Machine
- Imporving the Frame Rate of the video captured by the camera
Google Summer of Code has been a fun and wonderful experience for me. I got the chance to work with some great people and getting hands on with a working project that is going to be used by people around the world is a wholesome experience all together. Many thanks to my mentor Evan W. Patton and David Kim for helping me throughout this summer for making my project working successfully. And to even help me solve the beginner's task. And also thanks to Susan Rati Lane for the support as well. I hope my project will make App Inventor easier and more convenient to use for everyone 😊.