Skip to content

Instantly share code, notes, and snippets.

@neel-makhecha

neel-makhecha/GSoC20-Work-Product.md

Last active January 12, 2021 20:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save neel-makhecha/35357892bcc16978c47f1dcfb7ac53f8 to your computer and use it in GitHub Desktop.
Save neel-makhecha/35357892bcc16978c47f1dcfb7ac53f8 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
GSoC20-Work-Product.md

Google Summer of Code 2020 Project for Catrobat - Documentation

By Neel Makhecha

About Project

During the last three months of Google Summer of Code, as a Student Developer, I have worked on an amazing project by Catrobat - Pocket Code on iOS (available on Android too, here). Mainly, my GSoC 2020 project for Catrobat consists of the following two parts:

  1. Developing Pen Bricks for Pocket Code on iOS.
  2. Updating and improving UI, UX and Usability across the Pocket Code app.

I have discussed each of these under different sections below and some other contributions that I have made during the GSoC period in the later sections.

1. Developing Pen Bricks

Pen Bricks is a category of bricks consisting of the following different bricks in Catrobat (Check it out on Catrobat Wiki):

  • Pen Down Brick
  • Pen Up Brick
  • Set Pen Size Brick
  • Set Pen Color Brick
  • Stamp Brick
  • Clear Brick

Pen Bricks allow users to draw in a running Catrobat scene by controlling the motion of a sprite or object from the scripts. These bricks were already available on Pocket Code for Android (also known as Catroid). Mainly, my goal was about developing the same for Pocket Code on iOS (also known as Catty).

1. Getting started

I started by implementing some methods and necessary data structures that will be useful later when implementing pen bricks and their functionalities.

I) Implementing update function for SpriteNodes

Links: Issue, Merged PR

Similar to the built-in update function of SKScene (from SpriteKit) I had to implement the update function for CBSpriteNode (subclass of SKSpriteNode). Similar to SKScene, it is supposed to be called in proportion to the frame refresh rate. The update function of SKScene is called automatically at every new frame, so generally 60 times in one second (or about every 16 milliseconds). As we might have multiple sprite nodes in a scene, it would be more efficient not to call update method of each and every sprite node whenever SKScene's update method is invoked. Hence, this method will be called for all sprite nodes present inside the scene on every alternate call of SKScene's update. So all the sprite nodes will be updated at every other frame or in an interval of 32 milliseconds.

II) Creating SpriteKitDefines

Links: Issue, Merged PR

This class contains all the SpriteKit related constants which will be later useful in implementing Pen bricks.

III) Implementing PenConfiguration

Links: Issue, Merged PR. Update: Issue, Merged PR

PenConfiguration is a data structure that stores various properties of pen. It contains following mutable properties:

  • penDown: A value of type indicating whether the pen is down or not.
  • size: A value with a private setter of type CGFloat that will store the raw size (to be used directly with SpriteKit) useful when drawing pen lines.
  • catrobatSize: This value is scaled to match with the size value for pen on Catroid. This value will be accessible to users and size will be updated based on catrobatSize and other factors.
  • sizeConversionFactor: This is a static constant value determined manually, which is useful in scaling the catrobatPosition into an appropriate raw size value.
  • screenRatio: The raw size (thickness) of a pen line also depends on another value called screenRatio, which is calculated inside the PenConfiguration initialiser. It is the ratio of the diagonal pixels of the screen in which the project was first created to the diagonal pixels of the screen of the current device. This ratio is useful so that the thickness of pen lines are consistent across displays with different resolutions.
  • color: This property of type UIColor will store the color of the pen line drawn.
  • previousPositions: It is basically an array of type SynchronizedArray<CGPoint> that stores the previousPositions of the spriteNodes when the penDown property is true. It will keep on removing the elements as the lines between them are drawn (This is implemented in drawPenLine method in CBSpriteNode).

IV) Implementing 'Pen feature'

Links: Issue, Merged PR. Update 1: Issue, Merged PR. Update 2: Issue, Merged PR.

The basic idea of the pen feature is quite simple. For any sprite node, when the penDown property of PenConfiguration evaluates to true, a line should be drawn following the movement of the node.

This is implemented by having a function drawPenLine which gets called from the update method of CBSpriteNode which is implemented earlier. It evaluates whether penDown property is true or not. If it is true, and the last element of the previousPosition is not equal to the current position, it will append the current position in the previousPosition array of PenConfiguration. If there's more than one element in the previousPosition array, pen lines will be drawn between each of them and are removed from the array.

2. Implementing PenDownBrick

Links: Issue, Merged PR

Pen Down Brick

First a new category of bricks is defined as we didn't have any pen bricks before. All the bricks in the same category is meant for a specific type of functionality and have similar properties, like brick color (In this case, dark green #305716).

How does PenDownBrick work? The PenDownBrick will signal the penDown property of PenConfiguration of the corresponding object (sprite node) when encountered in the catrobat code. Once encountered, following every movement of that object (change in position) pen lines will be drawn on the scene as per the penConfiguration properties. This brick is only accessible for the scripts of 'objects' and not for the scripts of 'background'.

The following general steps were followed when implementing PenDownBrick (and other bricks too):

  1. Implementing a brick class called PenDownBrick that implements BrickProtocol and registering it in CatrobatSetup
  2. Implementing a new class called PenDownBrickCell which implements BrickCell and BrickCellProtocol. PenDownBrickCell is small in size and only has one label with a localized string.
  3. Defining instructions inside PlayerEngine/Instructions that get executed when any Pen bricks are executed. In this case, first, implementing the CBInstructionProtocol and then defining instructions for PenDownBrick in a method called actionBlock and running it from SKAction.
  4. Defining serialization for PenDownBrick by implementing the CBXMLNodeProtocol and implementing the parse function (serialization to object) and xmlElement function (object to serialization).

Serialization of PenDownBrick:

<brick type="PenDownBrick">
  <commentedOut>false</commentedOut> 
</brick>
  1. Test Case PenDownBrickTests is defined with a unit test testPenDownBrick which tests the instructions of PenDownBrick.

3. Implementing PenUpBrick

Links: Issue, Merged PR

Pen Up Brick

As the functionality of these bricks are self explanatory from their names, The purpose of PenUpBrick is exactly opposite to that of PenDownBrick. It will not have any significant effect when PenUpBrick is placed above PenDownBricks or it is not paired with a PenDownBrick which is placed before it in the Catrobat scripts. Similar to PenDownBrick, PenUpBrick also cannot be accessed from the scripts of 'background'.

Similar to PenDownBrick, implementing PenUpBrick was also done in the following steps:

  1. Implementing a brick class called PenUpBrick that implements BrickProtocol and registering it in CatrobatSetup
  2. Implementing a new class called PenUpBrickCell which implements BrickCell and BrickCellProtocol. PenDownBrickCell is small in size and only has one label with localized a string.
  3. Defining instructions that will get executed whenever PenUpBrick is encountered by implementing the CBInstructionProtocol in PenUpBrick and then defining instructions for PenUpBrick in a method called actionBlock and running it from SKAction.
  4. Defining serialization for PenUpBrick by implementing the CBXMLNodeProtocol and implementing the parse function (serialization to object) and xmlElement function (object to serialization).

Serialization of PenUpBrick:

<brick type="PenUpBrick">
  <commentedOut>false</commentedOut> 
</brick>

Test case PenUpBrickTests is defined with a unit test testPenUpBrick which tests the instructions of PenDownBrick.

4. Implementing 'clear' functionality and PenClearBrick

Links: Clear functionality: Issue, Merged PR. PenClearBrick: Issue, Merged PR.

A new method is implemented in CBScene (now renamed to Stage, subclass of SKScene) named clearPenLines which, in order to remove all the pen lines from the scene, removes all the nodes of custom type LineShapeNode (subclass of SKShapeNode) named SpriteKitDefines.penShapeNodeName. A new test has been added to test this functionality in SceneTests (now renamed to StageTests)

Pen Clear Brick

Similar to PenDownBrick and PenUpBrick, PenClearBrick is also implemented following the same steps, as it resembles many properties of those bricks, i.e., no specific input of any form is required, no pairing is required, etc. There is one difference between PenDownBrick (and PenUpBrick) and PenClearBrick, that is, PenClearBrick can be accessed from 'object' scripts as well as 'background' scripts. Because, ultimately the instructions of this brick will clear all the pen strokes from the scene regardless of which object drew it. So it is not necessary that it must be associated with any object.

The serialization is something that I will highlight, as it is a bit different from the other two bricks as follows:

<brick type="ClearBackgroundBrick">
  <commentedOut>false</commentedOut> 
</brick>

Here, notice that the type of brick is called ClearBackgroundBrick instead of PenClearBrick (same name as brick name) to make it consistent for the projects created on Catroid. This was discovered later in this bug and was fixed in this PR.

5. Implementing SetPenSizeBrick

Links: Issue, Merged PR.

Set Pen Size Brick

SetPenSizeBrick (as its self explanatory name again suggests) is useful to set the pen size (or thickness of pen lines). Implementing this brick was different than any other bricks that worked on previously. That's because this brick required an input field as well, where users can type in a value (or a formula) for the pen thickness. Not only that, the pen size should be uniform across devices with different screen size or resolution and with Catroid. This is already addressed on how it is implemented earlier under Implementing PenConfiguration title.

Not surprisingly, this change, i.e., the way this brick works lead to changes in many ways it is implemented. Now, instead of having a SetPenSizeBrick class that conforms to BrickProtocol, it should also implement BrickFormulaProtocol. This lead to implementing functions like formula, setFormula, getFormulas, etc. The formulaInterpreter casts the input value to float and will be assigned to the catrobatSize property of PenConfiguration. Hence, this brick will only accept one formula which can result in a float value.

The following is the serialization for SetPenSizeBrick, much different than previously implemented bricks. This also includes serialization of formula, its value and type which is implemented in a much similar way by conforming SetPenSizeBrick to CBXMLNodeProtocol.

<brick type="SetPenSizeBrick">
  <commentedOut>false</commentedOut>
  <formulaList>
    <formula category="PEN_SIZE">
      <type>NUMBER</type>
      <value>3.15</value>
    </formula>
  </formulaList>
</brick>

6. Implementing SetPenColorBrick

Links: Issue, Merged PR.

SetPenColorBrick is useful to set the color of the pen lines that will be drawn after the brick is encountered. Users will pass three integer values for (or formula that results to that value): red, blue and green. This is much similar to the implementation of SetPenSizeBrick with just a change that we are having three input text fields arranged as shown in the following brick image.

Set Pen Color Brick

Based on the values passed, a new value of UIColor will be assigned to the color property of the PenConfiguration object from the brick instructions. The serialization of the brick is as follows:

<brick type="SetPenColorBrick">
  <commentedOut>false</commentedOut>
  <formulaList>
    <formula category="PEN_COLOR_RED">
      <type>NUMBER</type>
      <value>0</value>
    </formula>
    <formula category="PEN_COLOR_BLUE">
      <type>NUMBER</type>
      <value>255</value>
    </formula>
    <formula category="PEN_COLOR_GREEN">
      <type>NUMBER</type>
      <value>0</value>
    </formula>
  </formulaList>
</brick>

7. Implementing StampBrick

Links: Issue, Merged PR

StampBrick, when encountered in the script will stamp the look of the object on the background (copy on the background) with the same position, zPosition, zRotation and alpha. Again, just like most other pen-bricks, StampBrick can only be used with objects (sprite nodes) and not with backgrounds. Just like PenDownBrick, PenUpBrick and PenClearBrick, StampBrick also follows the same steps for implementation: Creating a model class StampBrick that implements BrickProtocol, CBInstructionProtocol and CBXMLNodeProtocol. Creating basic StampBrickCell with only one label with localized string.

There is no input required from the users and hence it has a very simple serialization:

<brick type="StampBrick">
  <commentedOut>false</commentedOut>
</brick>

2. Working on UI/UX and Usability Improvements

1. Preview Image For Last Opened Project

Links: Issue, Merged PR

As a part of redesigning the landing page of the app, I have implemented this change where instead of showing a default continue project icon, it shows the project preview next to its name with some other minor changes.

This update led to many other changes as well. First, there is a new method added in the CBFileManager class called loadPreviewImageAndCache which takes two parameters of type ProjectLoadingInfo and a completion handler: (_ image: UIImage?, _ path: String?) -> Void. Based on the determined project path from the projectLoadingInfo argument, it is first looked up into the imageCache (property of type RuntimeImageCache). If not found from imageCache, only then it will read the image data from disk and load it into imageCache for faster access later. If the preview image is found, the completion handler is called with the image and its path passed as arguments, otherwise called with nil as arguments. The preview image is the screenshot of the project (either automatic or manual). In the case where the UIImage object is nil, the landing page will show the default icon.

The CatrobatTableViewController (landing page) is updated and now uses this method to show project preview. Moreover, MyProjectsViewController, which already showed preview images of all projects, is also updated, making use of this method. New tests are added to test this method thoroughly in CBFileManagerTests covering all possibilities using test doubles.

2. Rounded Corners for Project Previews Across the App:

Links: Issue, Merged PR.

The project previews across the app now have rounded corners, giving it an elegant look. The changes has been made on:

  • ChartProjectCell which shows project previews with names for the top charts projects in Catrobat Community
  • SearchStoreCell shows projects with preview when a project name is searched in Catrobat Community.
  • MyProjectsViewController

3. New Icons for Landing Page:

Links: Issue, Merged PR

Icons of the landing page are updated with the new SFSymbols. For now, they are imported as image assets to continue app compatibility on older versions of iOS.

4. Default Screenshots (Previews) for Newly Created Projects

Links: Issue, Merged PR

Whenever a new project is created, a random image from six available default images will be selected and assigned to the newly created project as an automatic screenshot. That way, there will be no project in future without any screenshots.

Various major changes has been made during the implementation of this:

A new class is created named ProjectManager which contains a static method createProject. The createProject method replaces the existing method defaultProjectWithName implemented in ObjC with few additions, that is, createProject will also select and add one of the 6 images as a default screenshot for newly created projects.

The older projects with no screenshot will show the first default image (out of 6) named 'catrobat'.

Along with these changes, a new test case is added named ProjectManagerTests that tests the newly implemented createProject method with test doubles. A new writeData method is also added inside CBFileManager to easily write Data on a specified path on disk.

Other Contributions

During the last three months of GSoC, other than the actual pre-planned project, I have also done a few other contributions. Like, refactoring the legacy Objective-C code base and converting it into Swift, fixing some bugs and implementing a deployment automation. I have highlighted those below:

Wrap-Up

The project as a whole is amazing and I truly enjoyed working on it. I loved the very concept of programming apps and games on smartphones using the idea of logically placing the bricks. I have learned a lot while working on it, especially some things that can hardly be learned from some solo personal projects or some school projects.

Special Thanks to my GSoC mentor Michael Herold. It was absolutely awesome to work with him.

Special Thanks to Catrobat for giving me the opportunity to work on this amazing project during Google Summer of Code 2020.

Special Thanks to Google. I really appreciate this program.

Connect with me on: LinkedIn, Twitter Visit: https://neelmakhecha.tech

@Raj-Dave368
Copy link

Sir, How did you Understand the CodeBase of This application?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment