Skip to content

Instantly share code, notes, and snippets.

@iFlameing
Last active August 24, 2019 16:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save iFlameing/cdc307c86fb4d8bceb35571f26d3998c to your computer and use it in GitHub Desktop.
Save iFlameing/cdc307c86fb4d8bceb35571f26d3998c to your computer and use it in GitHub Desktop.

Final Report For Google Summer Of Code 2019

GatsbyJs Preview For Plone CMS

Gatsby is a blazing fast static site generator for React. gatsby-source plugin is a Gatsby plugin that pulls the whole content tree from the plone.restapi and makes it available for querying via GraphQL in a hierarchical data structure, handling all native content types including images and files.Now gatsby-source-plugin has a new feature by which user can see the instant update of site whenever they make changes to the Plone CMS. gatsby-source-plugin supports all features of live updates i.e Create mode, modified mode and Delete mode. If you make any these event in Plone CMS you can see instant update into your hosted site.

gatsby-source-plugin has all the documentation we needed to get started with your wonderful project. We also have gatsby-starter-plone and gatsby-theme-plone . Currently gatsby-theme-plone is in alpha phase.

Get the code

The plugin is used to pull data from a Plone site and respond to all the websocket event i.e created, modified and delete to a Gatsby project , gatsby-source-plone: https://github.com/collective/gatsby-source-plone. Using the live preview features of gatsby-source-plone developed in this project require a special build of Plone CMS, which is currently available as a Docker image as defined at https://github.com/collective/gatsby-source-plone/blob/master/demo/docker-compose.yml.

Live Demos

Documentation for gatsby-source-plone (Gatsby site generated from mock Plone site): https://collective.github.io/gatsby-source-plone/

GatsbjJs Preview

Features

  • created whenever you create any file on Plone CMS it will automatically added to your hosted website.
  • modified whenever you update the content on Plone CMS it will automatically update the content on your hosted website.
  • delete whenever you delete the content on Plone CMS, content will be removed from your hosted website.

Future Plans

PUll Request

week 1st

What did I do this week?

During my first week of coding period, I implemented a websocket server in order to communicate with gatsby-source-plone and updated the development readme.md. In this continuing week I established a websocket server and connected the websocket server to the gatsby-source-plone. I enhanced my knowledge in this process, I learnt how to create a websocket and implemented a basic websocket API.

I also created a 'basis chat app' while learning 'socket.io'.

You can find the Pull Request(PR) details below.

collective/gatsby-source-plone#176

collective/gatsby-source-plone#177

What Coming Next?

  • In following week I am going to implement the add node to the gatsby-source-plone.

  • Read the api provided by gatsby-source-filesystem.

  • How gatsby-source-filesystem work behind the scene.

Did I get stuck anywhere?

Not particularly, this week generally have to work with socket implementation. I have to figure out how socket.io api works with express other than that, I have a great time working on project.

Week 2nd

What did I do this week?

In this week, I mostly spent my time reading the api of gatsby-source-filesystem and its implementation. Since I also have to support the preview feature as gatsby-source-filesystem does, So most of concept I could learnt from it. It also one of the requirements of my project. But making the API as gatsby-source-filesystem is not trivial one because gatsby-source-fileystem is based on file changes and it works on watch event of file change. Where as Plone is a headless CMS. To implement its feature we integrated the plone with the websocket and it is upcoming feature of Plone which is relaeased in version 5.3 but my mentor provided the docker image before its release, for this I am very thankful to my mentor @datakurre. My project consist of three milestones and I started with the first one adding preview for the added fields. You can see my pull request at

collective/gatsby-source-plone#181.

What is coming up next?

The Pr related to first milestone needs modification. It is just a initial commit regarding the first feature.

  • Some small issue will be still to resolved.
  • Refactoring the code and utilise the previous code.
  • Figuring out how to create breadcrumbs node and navigation node.

Did I get stuck anywhere?

Reading codebase written by others is not trivial one. Reading the API of gatsby-source-filesystem initially was a big headache but when I started reading this it became very simple. After understanding some API I am able to create the initial commit regarding the first milestone. Other than this I have a great time working on the project :)

Week 3rd

What did I do this week?

In this week I started from where I left i.e adding create node for the added field in Plone CMS. After some minor changes and with the discussion with mentor the pr got merged and My first milestone which is first in three is completed. This is just POC of the milestone which is modified for the long term after I achieved all the milestone as mentioned in the proposal. After achieving the first milestone I started with second one i.e adding node for the modified field in Plone CMS. I created a initial pr regarding this milestone in this week which is under process. This pr generally contains a feature i.e when user make any changes in Plone backend the gatsby automatically knows that and create node for the updated field and user sees these changes without rebuild. This is handy feature when you are developing continuously and want to see the project how it looks in real world.

collective/gatsby-source-plone#188

What is coming up next?

The above pr is related to second milestone and it need some modification before it merge into master. So my first priority is to merge this pr in coming week. Along with this

  • Read Node Api of gatsby.
  • Read codebase of gatsby-source-filesystem how they implemented the delete feature with filesystem.

Did I get stuck anywhere?

I got stuck during the implementation of updating node related to folder section. In which if we change the parent node we also have to modified the all the children associated with it. I stuck for around a day solving this problem and finally I solve with some change in approach as I am doing in subsequent changes.

Week 4th

What did I do this week?

I started from where I left i.e Implementing the left modification part in previous week. After some discussion with datakurre I am able to implement it. Then I started with my third milestone i.e adding preview for the deleted node in Plone CMS. I implement the intial functionality in two days but I am stuck at the position where I have to handle some exception. But I am able to resolve this issue at the end of the week and submitted the pr.

What is coming up next?

Some small issue are left in MVP. I will try to solve it as soon as possible so that we can announced this feature in community.

  • Refactoring some code
  • Removing logging api

Did I get stuck anywhere?

During the deletion of node of folder I got successive request from the PLone CMS for the deleted node. I have to handle the exception which will arise when I fetch the url which is deleted. It takes some time to figure out but finally I am able to solve this.

Week 5th

Hooray! I passed the first evaluation and now focused on 2nd evaluation.

What did I do this week?

I have completed all my milestone which is listed in timeline. I implemented the add, update and delete functionality in gatsby-source-plone but this is just MVP. I need to do some refactoring, writing test and adding documentation for the added feature. Also I have to improve the code quality but I think I have enough time for this. So, during the evaluation period I figured out some small issue and made pr for i.e Adding node based on permission provided by PloneCMS. And also made initial commit regarding scheduling Plone collection.

What is coming up next?

Currently I am working on Scheduling Plone collection i.e We have to update the PloneCollection node after some time of websocket event. So, In upcoming week I will try to implement this feature and MVP of gatsby-source-plone is complete.

Did I get stuck anywhere?

During this week I have to change the concept of adding modified node because Permission changes also fired modified event other than that I didn't stuck anywhere.

Week 6 - 7th

What did I do this week?

In this week I started writing the test for the gatsby-node API. I read lot's of documentation regarding jest because our previous test is written using this framework. I also do some research regarding the unit test and converting our whole source code into typescript. But we came to the conclusion that first, we write a unit test for all the function and if time permits we devote our entire time working rewriting whole source plugin. I have written a test for makecontentNode function which converts the fetched content into Gatsby Node. You can see the pr below

collective/gatsby-source-plone#200

What is coming up next?

I will try to test all the code which is written in gatsby-node.js. so, you can accept more test to come in next week.

Did I get stuck anywhere?

I faced some difficulty regarding setting up mock data for the test but fetching from the Plone CMS API I am able to gather the test data. Other than that I am having a good time :)

Week 8th

What did I do this week?

I have completed MVP in starting week of 2nd Phase, I moved on writing test for the added feature. I have to read lot's of documentation of jest and Plone CMS API for writing test and setting mock up data. I have written test for may function which is used for creating Gatsby Node from fetching data from Plone CMS. I also remove some redundant code which is not used more in gatsby-node.js file. You can see my pull request by visiting below link.

collective/gatsby-source-plone#200

collective/gatsby-source-plone#201

collective/gatsby-source-plone#204

collective/gatsby-source-plone#208

collective/gatsby-source-plone#214

What is coming up next?

One or more test Which is left in the code base. After that every line is tested and all code is cover with test. The test for breadcrumbs node is left and is completed in next one or two days :)

Did I get stuck anywhere?

Writing test for async code and setting the mock behaviour of fetching data is a tedious task. I have to manipulate some function to pass the test. It is most intense task which I have done in my gsoc period. But after reading the documentation of jest and some googling I am able to write the test. Other than this I am having a great time :)

Week 9th

What did I do this week?

Hoorya!! I passed the second evaluation. In the evaluation week I mainly completed the only test which is left before the second evaluation i.e Adding test for the ploneGenerator method which generate all the Gatsby Node initially. It is the most important generator function. I never tested the generator function in my life. I searched what is the best way to test the generator function and I found some response and successfully Send a pr for that. After that there is simple bug i.e Foreach array method randomly calling the api but we need syncrounus behaviour. After devoting one day and lot of thinking I found that only replacing the foreach method with for loop does the trick. Then I successfully created the pr for that also. You can see the pr link below.

collective/gatsby-source-plone#218

collective/gatsby-source-plone#225

What is coming up next?

Upto this time I have successfully implemented the MVP for my project. Written test for all the function which is present in the source code. Now, I only have to refactor the code and write documentation for the added feature. So, You can think that this period mainly deal with refactoring and documenting.

Did I get stuck anywhere?

Really for first time in this project I suffer with simple issue which is converting the foreach method to for loop. It was not easy to find what's going wrong with foreach. Why it doesn't behaviour synchronously. But after lot of thinking and experiment I am able to attend the behaviour i.e synchronous.

Week 10th

What did I do this week?

So, As I previously mention in this phase I have to write documentation and refactor the code. So, I started with writing documentation for the config options provided by the plugin. I have written all the config option properties which you can pass to plugin and extended your website feature. After adding documentation I started refactoring the code. I have flatten the code by creating separate function for each event I have added. I added the create, modified event and delete event. I refactor the code related to createEvent. You can see both the pr below.

collective/gatsby-source-plone#231

collective/gatsby-source-plone#229

What is coming up next?

So, I think that documentation part is over because I have created a new file for config options which covered all the information which I have made changes. I am currently working on refactoring deleteEvent and after that modified options. So, you can see lot of refactoring of code in coming weeks.

Did I get stuck anywhere?

During start of refactoring I stuck at a issue of not finding function and related stuff you can see on the pr but I find it out and able to solve it. Other than that I am having a good time :)

week 11 - 12th

What did I do this week?

During the last days of GSOC period I have to refactor code and have to reolve some small issue which I ahve to fixed. I started with refactoring I have refactor the code for all the feature. I flatten the codebase By splitting the larger function into smaller function. I repeat this function mainy time and able to enhance the readability of codebase. Then I got started with changing our custom logging method wiht gatsby reporter API. I read the docs and replaced our custom loggin method with it. We are missing the debug method from the reporter API. I asked gatsby community about it and they told me to open the issue regarding this feature. Most intresting part comes now. I have to refactor all the function and method to its respective folder so, that we can transform this plugin from js to typescript. I have to debug a lot, but thanks to test it saves lot of time whenever I miss something test failed, I go to that function and fixed the issue. Currently I am happly that all the issue related to GSOC is closed before a week and I also completed the issue regarding the extended goal.

What is coming up next?

I am keep contributing to this project and others project of Plone. I will be happy to become the part of Plone community.

Did I get stuck anywhere?

I stuck at many point during the implementation of these feature but I am able to overcome at every time. These 3Months are my one of the best days of my life. I am solving each days and learn something new.

Thanks for reading this. If you are intrested talking to me please DM me on twitter @iFlameing
@iFlameing
Copy link
Author

@datakurre thanks for the review :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment