During the last three months of Google Summer of Code, as a Student Developer, I have worked on an amazing project by Catrobat - Pocket Code on iOS (available on Android too, here). Mainly, my GSoC 2020 project for Catrobat consists of the following two parts:
- Developing Pen Bricks for Pocket Code on iOS.
- Updating and improving UI, UX and Usability across the Pocket Code app.
I have discussed each of these under different sections below and some other contributions that I have made during the GSoC period in the later sections.
Pen Bricks is a category of bricks consisting of the following different bricks in Catrobat (Check it out on Catrobat Wiki):
- Pen Down Brick
- Pen Up Brick
- Set Pen Size Brick
- Set Pen Color Brick
- Stamp Brick
- Clear Brick
Pen Bricks allow users to draw in a running Catrobat scene by controlling the motion of a sprite or object from the scripts. These bricks were already available on Pocket Code for Android (also known as Catroid). Mainly, my goal was about developing the same for Pocket Code on iOS (also known as Catty).
I started by implementing some methods and necessary data structures that will be useful later when implementing pen bricks and their functionalities.
Similar to the built-in update function of SKScene (from SpriteKit) I had to implement the update function for CBSpriteNode (subclass of SKSpriteNode). Similar to SKScene, it is supposed to be called in proportion to the frame refresh rate. The update function of SKScene is called automatically at every new frame, so generally 60 times in one second (or about every 16 milliseconds). As we might have multiple sprite nodes in a scene, it would be more efficient not to call update method of each and every sprite node whenever SKScene's update method is invoked. Hence, this method will be called for all sprite nodes present inside the scene on every alternate call of SKScene's update. So all the sprite nodes will be updated at every other frame or in an interval of 32 milliseconds.
This class contains all the SpriteKit related constants which will be later useful in implementing Pen bricks.
Links: Issue, Merged PR. Update: Issue, Merged PR
PenConfiguration is a data structure that stores various properties of pen. It contains following mutable properties:
penDown
: A value of type indicating whether the pen is down or not.size
: A value with a private setter of typeCGFloat
that will store the raw size (to be used directly with SpriteKit) useful when drawing pen lines.catrobatSize
: This value is scaled to match with the size value for pen on Catroid. This value will be accessible to users andsize
will be updated based oncatrobatSize
and other factors.sizeConversionFactor
: This is a static constant value determined manually, which is useful in scaling thecatrobatPosition
into an appropriate raw size value.screenRatio
: The raw size (thickness) of a pen line also depends on another value calledscreenRatio
, which is calculated inside the PenConfiguration initialiser. It is the ratio of the diagonal pixels of the screen in which the project was first created to the diagonal pixels of the screen of the current device. This ratio is useful so that the thickness of pen lines are consistent across displays with different resolutions.color
: This property of typeUIColor
will store the color of the pen line drawn.previousPositions
: It is basically an array of typeSynchronizedArray<CGPoint>
that stores the previousPositions of the spriteNodes when thepenDown
property is true. It will keep on removing the elements as the lines between them are drawn (This is implemented indrawPenLine
method inCBSpriteNode
).
Links: Issue, Merged PR. Update 1: Issue, Merged PR. Update 2: Issue, Merged PR.
The basic idea of the pen feature is quite simple. For any sprite node, when the penDown property of PenConfiguration evaluates to true, a line should be drawn following the movement of the node.
This is implemented by having a function drawPenLine
which gets called from the update method of CBSpriteNode which is implemented earlier. It evaluates whether penDown property is true or not. If it is true, and the last element of the previousPosition is not equal to the current position, it will append the current position in the previousPosition array of PenConfiguration. If there's more than one element in the previousPosition array, pen lines will be drawn between each of them and are removed from the array.
First a new category of bricks is defined as we didn't have any pen bricks before. All the bricks in the same category is meant for a specific type of functionality and have similar properties, like brick color (In this case, dark green #305716).
How does PenDownBrick work? The PenDownBrick will signal the penDown property of PenConfiguration of the corresponding object (sprite node) when encountered in the catrobat code. Once encountered, following every movement of that object (change in position) pen lines will be drawn on the scene as per the penConfiguration properties. This brick is only accessible for the scripts of 'objects' and not for the scripts of 'background'.
The following general steps were followed when implementing PenDownBrick (and other bricks too):
- Implementing a brick class called
PenDownBrick
that implementsBrickProtocol
and registering it inCatrobatSetup
- Implementing a new class called
PenDownBrickCell
which implementsBrickCell
andBrickCellProtocol
. PenDownBrickCell is small in size and only has one label with a localized string. - Defining instructions inside PlayerEngine/Instructions that get executed when any Pen bricks are executed. In this case, first, implementing the
CBInstructionProtocol
and then defining instructions for PenDownBrick in a method calledactionBlock
and running it from SKAction. - Defining serialization for PenDownBrick by implementing the
CBXMLNodeProtocol
and implementing theparse
function (serialization to object) andxmlElement
function (object to serialization).
Serialization of PenDownBrick:
<brick type="PenDownBrick">
<commentedOut>false</commentedOut>
</brick>
- Test Case
PenDownBrickTests
is defined with a unit testtestPenDownBrick
which tests the instructions of PenDownBrick.
As the functionality of these bricks are self explanatory from their names, The purpose of PenUpBrick is exactly opposite to that of PenDownBrick. It will not have any significant effect when PenUpBrick is placed above PenDownBricks or it is not paired with a PenDownBrick which is placed before it in the Catrobat scripts. Similar to PenDownBrick, PenUpBrick also cannot be accessed from the scripts of 'background'.
Similar to PenDownBrick, implementing PenUpBrick was also done in the following steps:
- Implementing a brick class called
PenUpBrick
that implementsBrickProtocol
and registering it inCatrobatSetup
- Implementing a new class called
PenUpBrickCell
which implementsBrickCell
andBrickCellProtocol
. PenDownBrickCell is small in size and only has one label with localized a string. - Defining instructions that will get executed whenever PenUpBrick is encountered by implementing the
CBInstructionProtocol
inPenUpBrick
and then defining instructions for PenUpBrick in a method calledactionBlock
and running it from SKAction. - Defining serialization for PenUpBrick by implementing the
CBXMLNodeProtocol
and implementing theparse
function (serialization to object) andxmlElement
function (object to serialization).
Serialization of PenUpBrick:
<brick type="PenUpBrick">
<commentedOut>false</commentedOut>
</brick>
Test case PenUpBrickTests
is defined with a unit test testPenUpBrick
which tests the instructions of PenDownBrick.
Links: Clear functionality: Issue, Merged PR. PenClearBrick: Issue, Merged PR.
A new method is implemented in CBScene
(now renamed to Stage
, subclass of SKScene) named clearPenLines
which, in order to remove all the pen lines from the scene, removes all the nodes of custom type LineShapeNode
(subclass of SKShapeNode) named SpriteKitDefines.penShapeNodeName
. A new test has been added to test this functionality in SceneTests
(now renamed to StageTests
)
Similar to PenDownBrick and PenUpBrick, PenClearBrick is also implemented following the same steps, as it resembles many properties of those bricks, i.e., no specific input of any form is required, no pairing is required, etc. There is one difference between PenDownBrick (and PenUpBrick) and PenClearBrick, that is, PenClearBrick can be accessed from 'object' scripts as well as 'background' scripts. Because, ultimately the instructions of this brick will clear all the pen strokes from the scene regardless of which object drew it. So it is not necessary that it must be associated with any object.
The serialization is something that I will highlight, as it is a bit different from the other two bricks as follows:
<brick type="ClearBackgroundBrick">
<commentedOut>false</commentedOut>
</brick>
Here, notice that the type of brick is called ClearBackgroundBrick
instead of PenClearBrick
(same name as brick name) to make it consistent for the projects created on Catroid. This was discovered later in this bug and was fixed in this PR.
SetPenSizeBrick (as its self explanatory name again suggests) is useful to set the pen size (or thickness of pen lines). Implementing this brick was different than any other bricks that worked on previously. That's because this brick required an input field as well, where users can type in a value (or a formula) for the pen thickness. Not only that, the pen size should be uniform across devices with different screen size or resolution and with Catroid. This is already addressed on how it is implemented earlier under Implementing PenConfiguration title.
Not surprisingly, this change, i.e., the way this brick works lead to changes in many ways it is implemented. Now, instead of having a SetPenSizeBrick
class that conforms to BrickProtocol
, it should also implement BrickFormulaProtocol
. This lead to implementing functions like formula
, setFormula
, getFormulas
, etc. The formulaInterpreter
casts the input value to float
and will be assigned to the catrobatSize property of PenConfiguration. Hence, this brick will only accept one formula which can result in a float value.
The following is the serialization for SetPenSizeBrick, much different than previously implemented bricks. This also includes serialization of formula, its value and type which is implemented in a much similar way by conforming SetPenSizeBrick to CBXMLNodeProtocol.
<brick type="SetPenSizeBrick">
<commentedOut>false</commentedOut>
<formulaList>
<formula category="PEN_SIZE">
<type>NUMBER</type>
<value>3.15</value>
</formula>
</formulaList>
</brick>
SetPenColorBrick is useful to set the color of the pen lines that will be drawn after the brick is encountered. Users will pass three integer values for (or formula that results to that value): red, blue and green. This is much similar to the implementation of SetPenSizeBrick
with just a change that we are having three input text fields arranged as shown in the following brick image.
Based on the values passed, a new value of UIColor will be assigned to the color
property of the PenConfiguration object from the brick instructions. The serialization of the brick is as follows:
<brick type="SetPenColorBrick">
<commentedOut>false</commentedOut>
<formulaList>
<formula category="PEN_COLOR_RED">
<type>NUMBER</type>
<value>0</value>
</formula>
<formula category="PEN_COLOR_BLUE">
<type>NUMBER</type>
<value>255</value>
</formula>
<formula category="PEN_COLOR_GREEN">
<type>NUMBER</type>
<value>0</value>
</formula>
</formulaList>
</brick>
StampBrick, when encountered in the script will stamp the look of the object on the background (copy on the background) with the same position, zPosition, zRotation and alpha. Again, just like most other pen-bricks, StampBrick can only be used with objects (sprite nodes) and not with backgrounds. Just like PenDownBrick, PenUpBrick and PenClearBrick, StampBrick also follows the same steps for implementation: Creating a model class StampBrick
that implements BrickProtocol, CBInstructionProtocol
and CBXMLNodeProtocol
. Creating basic StampBrickCell
with only one label with localized string.
There is no input required from the users and hence it has a very simple serialization:
<brick type="StampBrick">
<commentedOut>false</commentedOut>
</brick>
As a part of redesigning the landing page of the app, I have implemented this change where instead of showing a default continue project icon, it shows the project preview next to its name with some other minor changes.
This update led to many other changes as well. First, there is a new method added in the CBFileManager
class called loadPreviewImageAndCache
which takes two parameters of type ProjectLoadingInfo
and a completion handler: (_ image: UIImage?, _ path: String?) -> Void
. Based on the determined project path from the projectLoadingInfo argument, it is first looked up into the imageCache
(property of type RuntimeImageCache
). If not found from imageCache, only then it will read the image data from disk and load it into imageCache for faster access later. If the preview image is found, the completion handler is called with the image and its path passed as arguments, otherwise called with nil as arguments. The preview image is the screenshot of the project (either automatic or manual). In the case where the UIImage object is nil
, the landing page will show the default icon.
The CatrobatTableViewController
(landing page) is updated and now uses this method to show project preview. Moreover, MyProjectsViewController
, which already showed preview images of all projects, is also updated, making use of this method. New tests are added to test this method thoroughly in CBFileManagerTests
covering all possibilities using test doubles.
The project previews across the app now have rounded corners, giving it an elegant look. The changes has been made on:
ChartProjectCell
which shows project previews with names for the top charts projects in Catrobat CommunitySearchStoreCell
shows projects with preview when a project name is searched in Catrobat Community.MyProjectsViewController
Icons of the landing page are updated with the new SFSymbols. For now, they are imported as image assets to continue app compatibility on older versions of iOS.
Whenever a new project is created, a random image from six available default images will be selected and assigned to the newly created project as an automatic screenshot. That way, there will be no project in future without any screenshots.
Various major changes has been made during the implementation of this:
A new class is created named ProjectManager
which contains a static method createProject
. The createProject
method replaces the existing method defaultProjectWithName
implemented in ObjC with few additions, that is, createProject
will also select and add one of the 6 images as a default screenshot for newly created projects.
The older projects with no screenshot will show the first default image (out of 6) named 'catrobat'.
Along with these changes, a new test case is added named ProjectManagerTests
that tests the newly implemented createProject
method with test doubles. A new writeData
method is also added inside CBFileManager
to easily write Data on a specified path on disk.
During the last three months of GSoC, other than the actual pre-planned project, I have also done a few other contributions. Like, refactoring the legacy Objective-C code base and converting it into Swift, fixing some bugs and implementing a deployment automation. I have highlighted those below:
- Convert
UserVariable
from Objective-C to Swift with tests: Issue, Merged PR - Adding tests for
BrickCellVariableData
andBrickCellListData
: Issue 1, Merged PR 1, Issue 2, Merged PR 2. - Fixed accidental dismiss/swipes in ColorPickerViewController (Bug): Issue, Merged PR
- Convert
NSString
extension to Swift: Issue, Merged PR - Refactor
NSData
extension: Issue, Merged PR - Migrated
UIWebView
toWKWebView
: Issue, Merged PR - Fixed complete test run crash because of faulty project: Issue, Merged PR
- Fixed 'Axes are not completely visible on iPhone 11': Issue, Merged PR
- Generating automatic localized screenshots for App Store using Fastlane: Issue, Open PR
The project as a whole is amazing and I truly enjoyed working on it. I loved the very concept of programming apps and games on smartphones using the idea of logically placing the bricks. I have learned a lot while working on it, especially some things that can hardly be learned from some solo personal projects or some school projects.
Special Thanks to my GSoC mentor Michael Herold. It was absolutely awesome to work with him.
Special Thanks to Catrobat for giving me the opportunity to work on this amazing project during Google Summer of Code 2020.
Special Thanks to Google. I really appreciate this program.
Connect with me on: LinkedIn, Twitter Visit: https://neelmakhecha.tech
Sir, How did you Understand the CodeBase of This application?