This week in snapcraft

ww27.md

Week 27 in snapcraft

Welcome to the weekly development notes for snapcraft! This covers work from July 03 until July 07 of 2017.

External contributions

First and for most, we extend a thank you from the team to all the external contributions we've seen this week, a special shout out to @evan and @ppisati.

Development work

Core

Ongoing dicussions on the forum and feature branches took place last week:

• Consensus has been reached on manifest generation for a snap. Branches in progress and this is the forum post where it was finalized in.
• API discussions ongoing on how plugins can notify the assets used was discussed on this forum post.

Integration

• Added bash completion support in the advent of support in snapd. PR: #1390
• Correct handling of I/O errors on file replication logic. This allows one to work with snapcraft in docker from OS X to build snaps. PR: #1369
• Ongoing, with consensus now on how to declare architecture support of a snap for builders (builder hint) on this forum post.

Containers

• Project assigned containers are now cleaned on snapcraft clean. PR: #1372.
• Use the correct uid on container mount for project assets. PR: #1385

Plugins

• Default targets added for powerpc, ppcel64 and s390x. There are default targets for all the supported targets now. PR: #1377
• Cross compilation support for the autotools and waf plugin were proposed.

Project testing and CI

• Testing refactors for readability and speed. PR: #1391.
• Applied a workaround for the download failures due to CDN issues in our testbed PR: #1386.

News and outreach

• Catch @kyrofa at the MOOS conference where he will be talking about how snaps and Ubuntu Core are a nice fit for the Mission Oriented Operating Suite http://oceanai.mit.edu/moos-dawg/pmwiki/pmwiki.php?n=Talk.08-Fazzari
• If you haven't seen the last Office Hour session, you can catch up and watch it here on youtube
• What was once a PR is now documentation on the travis documentation on how to use snaps in your .travis.yml https://docs.travis-ci.com/user/installing-dependencies/#Installing-Snap-Packages
• One of our team members likes blockchains and got snapcraft.yamls merged upstream with ongoing tasks to enable CI/CD for bcoin, pyethapp and solidity.

ww28.md

Week 28 in snapcraft

Welcome to the weekly development notes for snapcraft! This covers work from July 10 until July 14 of 2017.

External contributions

First and for most, we extend a thank you from the team to all the external contributions we've seen this week, a special shout out to Evan Dandrea (@evan) and Rex Tsai (@chinchun).

Developments in master

• Improvements in the file collision detection in parts PR: #1405
• Set the file to record to manifest.yaml PR: #1406
• Improve the testing documentation to ease collaborator induction PR: #1394
• Testing fix to not break a users bzr configuration options PR: #1357
• Ensure cleanbuild with the --debug option enters a debug shell on failures PR: #1404
• Documentation improvements to the python plugin with better explanation of the dependency links attribute PR: #1402
• Mock improvements to the rustup cross compilation tests PR: #1396
• Cross compilation support in autotools PR: #1383
• Testing improvement to filter tests when running inside a docker container PR: #1375
• Determenistic pip list output to json in the python plugin PR: #1393
• Python 3.6 testing fixes PR: #1398
• Typo fixes in the python plugin documentation PR: #1401

Current Pull Requests

• Revert the CDN workaround in tests that caused connection resets as the CDN provider has fixed the root cause PR: #1410
• Setup XDG_CACHE_HOME for container builds PR: #1408
• Record the original snapcraft.yaml used for the build PR: #1407
• Check if there is a container to remove before trying to do so PR: #1403
• Better messaging between a missing LXD installation and a misconfigured one PR: #1400
• Ruby plugin support PR: #1399
• waf crosscompilation support PR: #1397
• Extract the rosdep logic from the catkin plugin PR: #1392

Active implementation forum discussions

• Defining how to track plugin assets
• Manifest file definition
• Support in the catkin plugin with python dependencies
• Organizing components for plugin dependencies
• Supporting cross compilation in plugins

Question and answers on the forum

• Cross compilation support with the python plugin
• Cross compilation with the jdk plugin
• Permissions wrong in meta/snap.yaml
• Support request for the cmake plugin
• git submodule help request
• Clarification for dependencies needed in a gtk3 application

Ecosystem

• Carla (@letozaf) writes about her experience in creating here first snap on her blog on this blog post and another one on creating a snap for a go cli application on this post
• Leo (@elopio) wrote about using single snap using gobot deployed onto many SOCs on this post and errbot, a snap for simplified chatops on this blog post.
• Christian (@kalikiana) wrote about cross compiling iota, a rust application, using snapcraft on his blog.

ww31.md

Week 31 in snapcraft

Welcome to the weekly development notes for snapcraft! This covers work from July 30 until August 6 of 2017.

Release

The release notes for `snapcraft 2.33 are out.

This release is availabe on the candidate channel of snapcraft, if already installed, the snap will be refreshed on its own, if not it is installable by running:

sudo snap install snapcraft --classic --candidate

Or installable with apt on the supported releases of Ubuntu (16.04 Xenial Xerus and 17.04 Zesty Zaurus) and the current development release of Ubuntu (17.10 Artful Aardvark) by running:

sudo apt install snapcraft

For the first time ever, snapcraft is also available on PyPI, it has been made installable on OSX and Windows, just run:

pip install snapcraft

on your python3 environment to get it, store commands and cleanbuild should work. That said, it has been made even easier on OSX as you can just:

brew install snapcraft

Development in master

• lxd: wait on lock files before running apt commands PR: #1435
• lxd: inject snapcraft and core snaps into the container PR: #1364
• core: cache FileBase entries when a checksum is provided PR: #1433
• lxd: clean with no parts should only delete PR: #1434

This weeks Pull Requests

• kbuild: support makefiles without install targets PR: #1432
• xd: wait on lock files before running apt commands PR: #1435
• cli: properly handle exceptions in lifecycle (an alternative proposal to PR: #1429)PR: #1436

Current active design forum discussions

• Snapcraft build-on hint for builders
• Plugin design for component organization
• kbuild and make install
• Persistent project containers updates

Ecosystem

goreleaser

• goreleaser, since v0.28.0, supports snaps as an output, take a look at their documentation on how to add snap support

They even publish a snap for goreleaser from goreleaser, check out their releases page

If you are on OS X, this together with brew install snapcraft can lead to an easy end to end story to get published on the Snap Store (more seemless integration coming soon). The same story is true for Windows users who want to pip install snapcraft.

Conferences

Kyle has just come back from presenting at the MOOS conference. We await his notes on the experience, but the general feedback is that people like snaps.

This month on the 18th and 19th, you can catch part of the snapcraft team, Leo and Sergio, speaking at the UbuconLA 2017, these are the sessions (they will be carried out in Spanish):

• snapcraft and the new free software communities
• Getting to your users
• Workshop on creating snaps and doing continous delivery through the snap store

ww32.md

Week 32 in snapcraft

Welcome to the weekly development notes for snapcraft! This covers work from August 7 until August 13 of 2017.

Development in master

The theme of code landing into master this week was mostly about robotics, from polishing and refactoring to ease the development of ROS 2 support (ament plugin) as well as supporting new features.

Aside from that, worth mentioning is that the next release of snapcraft (2.34) will have support for a Windows msi to ease the installation and get snapcraft working on that operating system (sans local building support).

• kbuild plugin: support Makefile without install target PR: #1432
• recording: record the original snapcraft.yaml PR: #1407
• options: properly handle missing compiler prefix PR: #1425
• kbuild plugin: move over the cross-compiling logic from the kernel plugin PR: #1417
• catkin plugin: include-roscore is a boolean PR: #1472
• catkin plugin: default to release build PR: #1470
• catkin plugin: rosinstall-files is a pull property PR: #1473
• catkin plugin: support passing args to cmake PR: #1471
• Extract rosdep into its own package PR: #1464
• Add cx_Freeze options targeting bin/snapcraft PR: #1478
• cli: better error message for missing mksquashfs PR: #1481
• ci: skip the CLA check for pull requests from the bot PR: #1482

This weeks Pull Requests

These are the pull requests that showed up during the week:

• lxd: configure user in container PR: #1484
• lxd: path cannot have extra forward slashes PR: #1483
• rosdep: add support for multiple dependency types PR: #1479
• kbuild plugin: use ARCH from environment if set PR: #1474
• add support for the "contact" field in snapcraft PR: #1447
• cli: properly handle exceptions PR: #1436

For a full list of pull requests go to https://github.com/snapcore/snapcraft/pulls

Current active design forum discussions

• Support the on statement for build-packages
• Running snapcraft as a unprivilidged user
• Updating persistent containers once and explicitly
• Recording build details from the python plugin
• Plugin design: how to organize components
• The meson plugin requires a specific target to exist
• Telling snapcraft not to generate wrappers

Ecosystem

More snaps!

If you needed to install these look no more and follow the installation instructions:

• geocoder: https://github.com/DenisCarriere/geocoder#snap-install
• internetarchive: https://internetarchive.readthedocs.io/en/latest/installation.html#snap

And some more snaps coming soon (blockchain related) with recently landed snapcraft.yaml and being added to ci/cd:

• solium: https://github.com/duaraghav8/Solium/blob/master/snap/snapcraft.yaml
• decentraland: https://github.com/decentraland/bronzeage-node/blob/master/snap/snapcraft.yaml

Conferences

This week, on the 18th and 19th, you can catch part of the snapcraft team, Leo and Sergio, speaking at the UbuconLA 2017, these are the sessions (they will be carried out in Spanish):

• snapcraft and the new free software communities
• Getting to your users
• Workshop on creating snaps and doing continous delivery through the snap store

ww33.md

Week 33 in snapcraft

Welcome to the weekly development notes for snapcraft! This covers work from August 14 until August 20 of 2017.

Development in master

Last week, error handling in snapcraft took the spotlight, this is the list of what happened in master:

• docs: fix typo in plugin help (PR: #1496)
• errors: use function for exit code (PR: #1491)
• ant plugin, gradle plugin: add support for authenticated proxies (PR: #1490)
• rosdep: add support for multiple dependency types (PR: #1479)
• lxd: path cannot have extra forward slashes (PR: #1483)
• meta: add base as a type and top level property (PR: #1419)
• many: use exceptions based on SnapcraftError PR: #1436
• cli: stop handling exceptions in store PR: #1436
• cli: stop handling exceptions in assertions PR: #1436
• cli: stop handling exceptions in parts PR: #1436
• plugins: use exceptions based on SnapcraftError PR: #1436
• cli: stop handling exceptions in lifecycle PR: #1436
• cli: add global exception handler PR: #1436

This weeks Pull Requests

List of open pull requests in the week that have not yet made it to master:

• lxd: pass original CLI arguments down to container PR: #1498
• ci: release snap to a branch for every PR PR: #1497
• cli: don't raise from excepthook PR: #1495
• Parameter forwarding for java object was blocked when using wrapper script PR: #1494
• lxd: only pass target arch if specified explicitly PR: #1493
• lxd: always remove existing device for project folder PR: #1488

Current active design forum discussions

• Syntax for build-snaps
• Cleanbuild remote on pi grabs wrong arch lxc image
• [Architecture handling for cross compilations](Specifying architecture on cross-compilations)

Ecosystem

New snaps

Carla Sella, a long time community member, has worked on a snap for creativecoin which she got integrated with https://build.snapcraft.io, effectively enabling CI/CD for it and can now be found on the edge channel.

Conferences

Leo and Sergio just came back from UbuconLA, here are their recorded presentations:

• snapcraft and the new free software communities
• Getting to your users
• Workshop on creating snaps and doing continous delivery through the snap store

ww34.md

Welcome to the weekly development notes for snapcraft! This covers work from 20 August until 26 August of 2017.

Development in master

• many: simplify plugin loading PR: #1507
• grammar: move into project_loader PR: #1500
• project loader: refactor into package PR: #1504
• ci: speedup the CLA check PR: #1503
• tests: use assertThat instead of assertEqual PR: #1501
• python plugin: record manifest PR: #1487
• repo: make errors based on SnapcraftError PR: #1499
• cli: don't raise from excepthook PR: #1495
• lxd: always remove existing device for project folder PR: #1488
• core: improve source caching logic PR: #1486

This weeks Pull Requests

• project_loader: process stage package grammar PR: #1509
• errors: introduce ContainerError PR: #1505
• lxd: pass original CLI arguments down to container PR: #1498

Current active design forum discussions

• Libraries not found in classic snaps
• Consume the build output of part 1 in part 2
• Suggestion: snapcraft init-gui
• Update persistent LXD containers
• ImportError: No module named six
• Week 33 of 2017 in snapcraft
• Snapcraft exits with 'dict' object is not callable
• How to use snapcraft to push snaps to the specific brand store？
• Syntax for build-snaps
• Snapcraft - my first snap issues
• Running snap as daemon with several instances
• Snapcraft PLUGIN -- jdk (Java runtime) ---> app Error
• Stack traces of snapcraft executable
• Use of content interface for sharing files (read-only)

Ecosystem

• Kyle has a guest post on electronics weekly
• Sergio, along with Alan, ran a snapcraft office hours featuring rust
• Sergio wrote down his thoughts about UbuconLA that took place this year in Lima, Peru on insights

ww36.md

Welcome to the weekly development notes for snapcraft! This covers work from 03 September until 09 September of 2017.

Development in master

• nodejs plugin: switch from --global-folder to --prefix PR: #1539
• jhbuild plugin: remove dependency on pkgconf PR: #1535
• tests: update the store failure tests PR: #1538
• jhbuild plugin: fix UnboundLocalError for chmod_path PR: #1534
• demos: update the name of the remote mqtt part PR: #1533
• tests: update the meson test to latest meson requirements PR: #1532
• lxd: use a unique temporary folder PR: #1519
• jhbuild plugin: new plugin PR: #1298
• project: introduce build-snaps PR: #1518
• repo: use os-release(5) to detect supported Linux distributions PR: #1527
• demos: remove the unnecessary wrapper from the java demo PR: #1494

This weeks Pull Requests

• tests: add integration tests for build snaps PR: #1542
• tests: fix the TEST_STORE environment variable PR: #1541
• plugins: extract python finder functions PR: #1540
• project_loader: aliases are deprecated PR: #1537
• repo: implement :target suffix for package names PR: #1536
• tests: share the cache dir in the integration suite PR: #1530
• recording: record the packages installed in the machine PR: #1529
• recording: record the machine information collected by uname PR: #1528

Current active design forum discussions

• Kivy App Not Working In Strict Confinement
• Launchpad now builds snaps in LXD containers, in support of "build-snaps"
• Easy configuration/customization for a snap
• Use of Python's DistUtilsExtra causes pip install failure
• Staging dependencies that go into the snap
• Having trouble understanding snap packaging
• Snapcraft snap doesn't exclude any libs from snaps it builds
• Specifying architecture on cross-compilations
• Week 35 of 2017 in snapcraft
• Snapcraft: easy way to know what is target arch in scriptlets
• 11 questions with no answers on Stack Exchange
• Snapcraft autopkgtest failure
• Why doesn't my nodejs snap create a binary?
• Snapcraft errors with No such directory /tmp/
• Run snapcraft in container as a user
• Qt5 open in browser (QDesktopServices::openUrl()) does't work
• Declaratively defining environment variables
• Snapcraft build-on hint for builders
• Asset recording for a built snap
• Where has snapcraft edge snap gone?

Ecosystem

ww45.md

Welcome to the weekly development notes for snapcraft! It has been a while since the last one, so we will quickly recap on some items from the past weeks as well.

Events

• Sergio presented Snaps for IoT at AllThingsOpen this past October.
• Kyle talked about Ubuntu, snaps, and Snapcraft at the ACM kick-off event/Ubuntu release party at Walla Walla University.
• This weekend Sergio will be presenting at PyConAR 2017.

New snaps

Some members of the team take some time to help around in the ecosystem, Leo has been focusing on these snaps these past weeks:

• parity, helping them to make it work fully confined and with the desktop assets, to prepare for the first stable release.
• ipfs-cluster has just been adopted by the upstream developer, but required two work arounds: $HOME in go and building with gx.
• vault was transferred to snapcrafters for better maintenance.
• yarn now works even with packags that require node-gyp. It's on discussion to get classic confinement, and then it will be ready for a call for testing.

Work in progress

During the past weeks the team has been working on consolidating what will be the 2.35 release, it is an interesting release all together, make sure to take a look at the (as of today) draft release notes.

Lot's of stabilization went into this release, the target is that this release is the one that will make snapcraft available on the stable channel. Asset recording is ready to collect detailed information from a clean container and 14.04 LTS could be used as a base operating environment to build snaps (through the snapcraft snap).

We have also been preparing the tasks for google code-in, to welcome new and young contributors into the project.

Keep in eye on the forum topic about the call for testing of 2.35 if you are at all interested.

Looking into the future

The team has mapped out the planned work items all the way through to 18.04; tasks have been sized by means of estimates to have a better idea of the workload the team can take. We invite you to take a look at the project and subscribe to the items of your interest.

The pool of tasks there will be selected on a weekly basis to make out the next milestone, which is 2.36

ww46.md

Welcome to the weekly development notes for snapcraft!

Events

Sergio presented a talk about snapcraft and the evolution of the python plugin at the PyConAR this past Sunday

Release 2.35

• A call for testing was made by @elopio https://forum.snapcraft.io/t/call-for-testing-snapcraft-2-35/2880
• The release notes have been finalized, we invite you to explore the new features, some of which are demonstrated in short videos https://github.com/snapcore/snapcraft/releases/tag/2.35
• snapcraft has made it to the stable channel

Release 2.36

Many new features have already landed for 2.36 which is due next Tuesday. Among them we have two new plugins: a new ament plugin required to build ROS 2 based projects and also a new catkin-tools plugin as an alternative to the catkin plugin for ROS.

Check out the status and tracking of items over at the milestone page.

Snaps

Some members of the team take some time to help around in the ecosystem, Leo has been focusing on getting ipfs and ipfs-cluster on being built and released for armhf using snapcraft cross building from travis.

ww47.md

Welcome to the weekly development notes for snapcraft!

Events

• Sergio is going to be presenting Ubuntu Core at the Córdoba IoT Meetup on Wednesday 6.
• The Google Code-in event has started; @kyrofa and @sergiusens have been having a good time so far as mentors.

Release 2.36

Release 2.36 has been tagged and currently in beta. The user oriented release notes will start showing up in the following days.

In the meantime, check out the work that went into this release over at the milestone page.

Release 2.37

With a reduced team this week as it is only @sergiusens and @kyrofa the 2.37 milestone will not be too feature rich but it will still have interesting and long awaited features rolled in such as proper rpath setting for prebuilt binaries when working with classic confinement and the new grammar for the architectures field for snapcraft.yaml.

ww48.md

Welcome to the weekly development notes for snapcraft!

Events

• Sergio is going to be presenting Ubuntu Core at the Córdoba IoT Meetup on Wednesday 6.
• The Google Code-in event has started; @elopio, @kyrofa and @sergiusens have been having a good time so far as mentors.

Code-In

The team wants to say thank you to all the Code-In participants so far, we have seen a plethora of contributions ranging from:

• fixing bugs in snapcraft picked up from the bug tracker.
• adding support for type hinting to some python modules.
• refactoring snapcraft code for better maintainability.
• proof reading tutorials.
• translating tutorials and videos to different languages.
• creating snaps out of python, node and ruby projects.

Release 2.37

The milestone for 2.37 has been closed and the release notes are in progress. The user oriented release notes will start showing up in the following days.

In the meantime, check out the work that went into this release over at the milestone page.

Some things were dropped from the milestone such as the architectures rework which would provide build hints for architectures as the team decided the design needed one more round of brainstorming in order to present the feature in a clear manner to its users.

That said, this release is not without its excitement, when creating snaps with classic confinement, snapcraft will do the right thing with regards to setting rpaths and the linker for prebuilt binaries (as long as building on a matching target base), allowing snapcrafters to get rid of the need to export LD_LIBARY_PATH which has its drawbacks when calling other binaries from the host.

Also, a lot of work has gone into UI polish, running snapcraft help is now sane and all the error messages have been consolidated with a pattern of "what went wrong, why it went wrong and how to fix". We really appreciate all of @mpt's input for the team to get there.

Another feature we want to mention is a great leap forward for control of snaps in CI: the new snapcraft export-login command, along with the login --with option. Want to upload snaps just to the edge channel from CI? You can now export a login that can do only that, and nothing more. Want to create a CI system that puts your edge snap through a gauntlet of tests, and promotes it to beta if they pass? You can export a login that can do only that. Check snapcraft export-login --help for more details.

One more cool feature comes from @matiasb (thank you!), and that's the addition of support for binary metadata (specifically icons right now) into snapcraft push-metadata.

Release 2.38

The 2.37 milestone has been planned and is good to go.

It will include the initial work on allowing to dump binaries from other bases to work correctly on the 16.04 base/core snap most developers target.

Also, long awaited, the ground works to allow for other sources of information to be consumed is going to get started in 2.38 which would allow adding handlers for consuming information from sources like appstream XMLs, setup.py, and similar.

On the cross compilation front, the on grammar is going to be expanded to also use from to clearly define behavior across different architectures when cross compiling.

ww49-50.md

Welcome to the weekly development notes, covering the past two weeks, for snapcraft!

Events

• Sergio presented Ubuntu Core at the Córdoba IoT Meetup on Wednesday 6.
• The Google Code-in event is ongoing; @elopio, @kyrofa and @sergiusens have been having a good time so far as mentors.

Code-In

The team wants to say thank you to all the Code-In participants so far, we have seen a plethora of contributions ranging from:

• fixing bugs in snapcraft picked up from the bug tracker.
• adding support for type hinting to some python modules.
• translating tutorials and videos to different languages.
• creating snaps out of python, node and ruby projects.

Release 2.38

Fresh out of the oven, 2.38 milestone has been closed, the code has been tagged and the release notes are currently being drafted.

The interesting tidbits are,

• full support for creating classic snaps from dumped binaries.
• grammar support for sources (you can now pick an architecture).
• a new plugin, catkin-tools.
• initial appstream support for information extraction (currently summary and description).

Docker image

A new docker image, built out of the snapcraft snap from edge is no available from dockerhub, to get it just pull from the edge tag.