The current gist represents the final report of my work on the GSoC 2019 project ”The missing features of NextCloudPi”.
The aim of this project is to extend NextCloudPi with additional features. NextCloudPi is an official project of Nextcloud, whose goal is to lower the barrier and make it easier for the end user to use Nextcloud. Baring in mind this goal, the current GSoC project focuses on automating tasks for NextCloudPi users and implementing missing features, based on new ideas or existing needs of the community. These new tools and features will not only offer end users more options, but will also help NextCloudPi become more prominent and enter the professional world.
All of my work can be found at the project's repository which was forked from the official nextcloudpi repository. My commits can be found at gsoc2019-devel branch, which contains exclusively my work (no mixed commits from the official repo), starting from commit d1496db.
Below, there is a list with the tasks that I successfully completed and some features that have been merged to the official repository. Each of the tools I created is explicitly documented at the README.md file of the repository. Moreover, I provide links to some articles on my blog about each task, which constitute useful tutorials to people who get started with the tools I used, explain the code I implemented step by step, using code flowcharts and screenshots of the execution.
-
CI/CD for NextCloudPi Docker images: The purpose of this task is to provide a way of transparent and automated building, testing and pushing to DockerHub of the NCP docker images. For this task, I used Travis CI. You can read more about the process of creating NCP's travis.yml and how I managed to overcome any problems on Travis here. Also, throughout this task I created some tools for automatically restarting jobs and clearing cache on Travis boxes, which are explained in this article.
-
NextCloudPi VM on VirtualBox easy setup and clone using Vagrant: This task focuses on setting up a VirtualBox VM through Vagrantfile and provide an easy way to clone it. I created an automation script that spawns a new NCP VM just by asking the user to provide a minimum input and can also create a clone of an existing NCP VM. People who are having a hard time to setup a VM on their own will benefit from this script as it does not ask for any technical details. This article on my blog explains the process and documents every step of the automation script.
-
Distributed system of NextCloudPi using Docker Swarm and GlusterFS: The aim of this task is to create a distributed system of NextCloudPi, that replicates the data storage to all nodes. I chose to use Docker Swarm as the container orchestrator of my system, which manages the nextcloudpi docker containers. I also used GlusterFS, a distributed software-based filesystem, in order to provide replication and data persistence of NCP data storage. I have implemented a tool that creates automatically the NCP Docker Swarm and configures the GlusterFS on each node. Other tools were also created providing extra features for the system, such as monitoring the system and adding new nodes. You can find an article I wrote about this task, explaining the whole process of the setup and documents the automation script as well as the extra tools step by step, posted at the official blog of NextCloudPi and also at my blog.
-
Ansiblize NextCloudPi server: According to this task, the server of NextCloudPi, which is written in bash scripts should be converted into Ansible playbooks. Although this task is not complete yet, I have implemented a big part of it, converting NCP-CONFIG sections SYSTEM and CONFIG into Ansible playbooks. You can read more about this task here.
-
Fix missing updates system: The goal of this task was for NCP to have more robust upgrades. I had to refactor the already existing bash scripts that implemented the updating system, so as to be more gentle with users skipping single updates by checking the current version and applying all historical upgrades one by one. I created the mechanism for this by implementing binary search on the current version of user's NCP, so as to find the right update script to begin with the updates, and get easily to the most recent version. My work on this task has been merged to the official nextcloudpi repository. You can see the pull request here.
-
Docker images fix: Another pull request that has been approved is this which fixes the names of the docker images that are being pushed and their tags, so that they correspond to the correct images.
- Complete Ansible's task: Sections CONFIG and SYSTEM are implemented. The rest sections should be converted also to Ansible playbook. When everything is converted, ncp-config should be updated using Ansible commands.
- Fix travis.yml to produce native NCP images as well: There was an issue with the Locales at the build phase of the native images. Should be resolved. Under the directory .travis/travis_instances/non-docker there are some yml files of the attempts used to produce native images.
- Create AppArmor profile with SecureWilly tool: An attempt was made to produce AppArmor profile for NCP but docker logs were not matching the template of SecureWilly. Needs investigation. Under the directory SecureWilly/Parser you can find the input_sample used as input to SecureWilly at this attempt.