GSoC 2019 - Final Assessment

assessment.md

GSoC 2019 - Final assessment

First of all, I would like to give huge thanks to my mentor, Tomáš Smetana, and Tom Barron and Goutham Pacha Ravi for all the technical help and decision making they provided me with during the project as well as before it, and hopefully after too. The same goes to the reviewers who gave me feedback and guidance, improving the quality of the end result.

The goals of this GSoC project were:

• implement snapshotting (and snapshot recovery - required by the CSI spec) features into the Manila CSI driver,
• currently, OpenStack Manila (Stein release) doesn't support creating shares from CephFS snapshots. The Manila CSI driver must therefore supplement this feature and work in a "compatibility mode" until such functionality is advertised by Manila,
• add unit and acceptance tests for Manila CSI

Work summary

Achieved goals:

• snapshotting and snapshot recovery is now available for all currently supported share protocols whose share types advertise snapshot_support and create_share_from_snapshot_support capabilities in their Extra Specs
• if create_share_from_snapshot_support for native CephFS is not advertised, a compatibility layers is used instead. Its internals are described here,
• integrated the CSI Sanity test suite (unit tests),
• added a Zuul CI acceptance job that tests the whole life-time of a share in a real cloud: create->snapshot->restore->mount->delete,
• added a Helm chart for Manila CSI.

Below are the respective GitHub posts that document the work:

Issues

1. kubernetes/cloud-provider-openstack#453 [RFE] Manila: add snapshot support
2. gophercloud/gophercloud#1653 Manila: add support for share metadata
3. kubernetes/cloud-provider-openstack#714 [Manila CSI] add csi-sanity tests
4. kubernetes/cloud-provider-openstack#720 [Manila CSI] add support for CephFS snapshots

Pull Requests

1. kubernetes/cloud-provider-openstack#652 csi-manila: added initial support for snapshots
2. kubernetes/cloud-provider-openstack#704 csi-manila: roll-back storage resources on failure
3. gophercloud/gophercloud#1656 Manila: add support for share metadata modification
4. gophercloud/gophercloud#1662 manila share-types: add IDFromName convenience function
5. kubernetes/cloud-provider-openstack#711 csi-manila: deployment manifests refactoring + helm chart
6. kubernetes/cloud-provider-openstack#716 [csi-manila] Manila client moved into a separate package
7. kubernetes/cloud-provider-openstack#728 [csi-manila] CSI client moved into a separate package
8. kubernetes/cloud-provider-openstack#730 [csi-manila] fixed incorrect behaviour discovered by csi-sanity test suite
9. kubernetes/cloud-provider-openstack#732 [csi-manila] added csi-sanity tests
10. theopenlab/openlab-zuul-jobs#624 Add csi-manila job
11. kubernetes/cloud-provider-openstack#736 [csi-manila] added a basis for compatibility layers

The commits below are finished but cannot be sent as Pull Requests just yet because they are being blocked by some other PRs that are still under review:

Commits done but waiting for PRs efab317 updated compatibility options 3ec3beb cephfs share adapter: fixed wrong scope of accessRight var 47bd354 cephfs share adapter: BuildNodeStageSecret must check for empty values 539008a Dockerfile: added rsync to manila-csi-plugin image 02ba32f added CephfsCreateShareFromSnapshot compat layer 5c4a9fd removed hard-coded failures for cephfs snapshots; added checks for snapshot capabilities 3e50112 updated Helm chart with cephfs compat settings 61c694b updated docs ac0fe58 added a Zuul job for csi-manila

Conclusion

I've managed to meet almost all of the goals except the snapshot recovery compatibility layer for NFS Ganesha (CephFS-over-NFS). Since this feature is already implemented for native CephFS, it should be trivial to add it for Ganesha later.

Some of the features (mentioned in the list above) have not yet been merged as they are blocked by some PRs that are still under review.

Future work

Manila CSI is far from feature complete. Some of the planned features/fixes are: rework the concurrency synchronization scheme, add health monitoring, availability zones support, volume resizing...