Skip to content

Instantly share code, notes, and snippets.

@Korusuke

Korusuke/ReadMe.md

Last active August 26, 2019 16:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Korusuke/7b2ccf20533bc904b18ec5554d149730 to your computer and use it in GitHub Desktop.
Save Korusuke/7b2ccf20533bc904b18ec5554d149730 to your computer and use it in GitHub Desktop.
Download ZIP
GSoC 2019 - Work Product
Raw
ReadMe.md

GSoC 2019 Work Product: Automate packaging process for MacPorts using upt

Overview

The idea of this GSoC project was to automate the tedious process of packaging and updating ports making it easier for MacPorts Maintainers to add and maintain ports. With the implementation of upt, it will replace tools currently used by MacPorts maintainers namely pypi2port and cpan2port. This project also focuses on adding features to UPT such as recursive packaging and auto-updates of packages.

Major Goals:

  • Improve the Macports Backend for upt

  • Create NPM Frontend for upt

  • Create Haskell Frontend for upt

  • Add Colored Output

  • Add Recursive Packaging feature to upt

  • Add Update Package feature to upt

  • Improve the MacPorts Backend:

Some of the backends were already written by @mojca, all that needed to be done was to write the templates for all frontends.

Due to the difference in license identifiers used by upt and MacPorts, there was a need to convert them so to use in the Portfile template. Apart from this, as the output was being written to stdout till now it was necessary to add an output to file feature to the backend which is an already implemented feature of upt. The output to file feature takes care of creating the hierarchy of folders as well as writing the Portfile itself.

Tons and tons of diff files were generated for making sure that all templates are correct and there are no lint errors.

  • Create NPM Frontend for upt:

MacPorts doesn’t have any npm ports except for node and npm itself. So the goal of this task was to add support for packaging npm packages. At first, this idea seemed feasible and a basic frontend was written for the same. But after a bunch of experimentation and research, we hit a few major roadblocks. NPM usually installs packages locally(just for that specific project) which is not supported by MacPorts. Apart from that, different npm packages have different dependency version requirements which many a time leads to version mismatch errors. To overcome this, MacPorts will need to have many different versions of the same packages at the same time.

Due to all of this, completing the npm frontend was ruled out of the scope of this project and is something to complete in the future.

  • Create Haskell Frontend for upt:

Haskell Frontend was started from scratch and is now in a usable state. The main issue faced was parsing of Haskell cabal description files. As Haskell doesn’t provide a usable API like other upstreams wherein we get package description in JSON or similar format we had to go a bit out of the way and decided to parse in the HTML page of the packages instead. Being an experimental frontend at this stage there are few things yet to be done which are up for discussion and will be completed soon.

  • Add Colored Output

Well, this was one of the easiest things to implement as it was just a matter of import colorlog and defining which colors to use. Looking back this was the only feature that required no research at all.

  • Add Recursive Packaging feature to upt:

This feature took a lot more time than initially expected but the final result was very good making all the efforts worth it. It went through many iterations and updates to upt as well as MacPorts backend code. Settling on a naming convention for MacPorts was also quite a bit of discussion.

  • Add Update Package feature to upt

This feature was known to be the trickiest from the very start and you can be sure it is. This is something that is still in the works and yet to be merged. After trying different strategies for a smart update as of now we have settled on using just regex magic.

Merged PR’s:

upt-macports:

upt-hackage:

upt-cpan:

upt:

macports-ports:

Open PR’s:

upt-macports:

upt-hackage:

upt:

Words of Thanks

I would like to thank my mentors @reneeotten and @Steap for their continuous guidance throughout the summer. I would also like to thank @mojca for getting me started with the project and all the guidance and help. Special thanks to everyone at MacPorts for all the help and awesome summer.

Conclusion

With this, we come to an end to a pleasant summer working with MacPorts and an amazing GSoC experience. GSoC has helped me get started with Open Source, a journey that has begun and will hopefully never end.

As GSoC 2019 is coming to an end and we have to submit a work product for the same to point out the exact work done during the period.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment