The plan for the project is to create an image editing / pixelation tool based on the following paper: External link to paper. This tool would allow users to create pixel art that can be used in numerous design ways. The manual process of creating pixel art from existing images is a time consuming process. Since there are many options in pixelation (number of pixels, length, and number of colors), it is important to allow users to set these parameters to their liking. By making these parameters saveable, users can change their work at a later time or see which parameters provide the best solution for their specific needs.
These are the goals we established with the company during the planning phase:
- To use the existing user interface with which the users are familiar.
- To change the size of the area
- Add the ability to undo the process and view the previous parameters for the algorithm.
- Backward compatibility on older devices
- Test-driven development
- Continuous testing and integration with the existing code base.
The two biggest challenges with this project were the reliability of the algorithm, as it is very computationally intensive, and the integration of the project into the existing code base. After a deep dive into the code, the reusability of the user interface made the most sense, as it doesn't change the user experience too much and the usability of the tool is intuitive. With the help of the mentors, who are knowledgeable about the code base and usability, we overcame this initial hurdle. The biggest challenge was integrating the mathematical part of the paper into the existing code base. Since the program is deployed on handhelds, we are constrained by the given computational power and the limited power of the device. To get around the computationally intensive background of the algorithm, we came up with a few things. Multithreading the computation on the CPU, or using shaders for the computation, but since it is an iterative process, there is an upper limit to the time saved by parallelizing the computations. In implementing the algorithm using shaders, I encountered challenges with the various APIs available. On Android 7.0 (Nougat), API level 24 or higher, there is support for Vulkan; on older Android versions, there is the OpenGL ES Api. Since backward compatibility is important, it is best to use the computations with multithreading.
The tool is in an experimental stage where more work is needed to make it operational and user-friendly. We are currently tuning the mathematical part of the project and looking for ways to increase speed, but not at the expense of accuracy. Once this is done, the next step is to integrate the tool into the production code.
The scope of the project went beyond the original vision, and the goal is to finish the code and move computation from the CPU to the GPU in the future. With the Catrobat team, we expect this to be the next step after the current implementation is deployed in production.
As the project progressed, we encountered many issues with the existing code base and integration of the project for backward compatibility. By gaining insight into usability and user interface design, I acquired knowledge in creating intuitive user touchpoints and experiences.
The processing of the Algorithm
The Ui for the Tool
An example of the pixelation in process Here is the original photo :
And here the same photo after the pixelation algorithm
