Apache Zeppelin has a helpful feature in its Spark Interpreter called Object Exchange. This allows you to pass objects, including DataFrames, between Scala and Python paragraphs of the same notebook. You can do your data prep/feature engineering with the Scala Spark Interpreter, and then pass off a DataFrame containing the features to PySpark for use with libraries like NumPy and scikit-learn. Also with Zeppelin's support for matplotlib you have a pretty good setup for poking around and testing out machine learning on your data.
The MIT License (MIT)
Copyright (c) 2014 David Underwood
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
A Dashing widget to display the number of issues returned by a Jira Query Language (JQL) search. The number of issues will be displayed using the Numbers widget. This widget is a generalized version of the Atlassian Jira: Number of issues in a Jira filter query widget, which shows the number of issues for saved Jira filter.
The following dependencies are required. Please add them to your dashing gemfile.
gem 'jira-ruby', :require => 'jira'
|# Dashing service|
|# Add this file to /etc/init.d/|
|# $ sudo cp dashboard /etc/init.d/|
|# Update variables DASHING_DIR, GEM_HOME, & PATH to suit your installation|
|# $ sudo nano /etc/init.d/dashboard|
|# Make executable|
|# $ sudo chmod 755 /etc/init.d/dashboard|
|# Update rc.d|
|# $ sudo update-rc.d dashboard defaults|
A Very Short Guide to Writing Guides
This is just a few thoughts on the topic of writing technical guides. This was intended for Basho's engineering team, but this may apply to open source projects in general.
It's commonly preached that the first step in writing is to identify your audience; to whom are you writing? This is the most well known, most repeated, and most overlooked step of writing in general and technical writing in particular. Take this document, for example. My audience is technical people who need to communicate technical information, and not teenagers, so I shy away from images of pop icons and memes. I use jargon and words like "identify" rather than "peep this".
|Vagrant::Config.run do |config||
|# Every Vagrant virtual environment requires a box to build off of.|
|config.vm.box = "base"|
|# Assing IP to the VM|
|config.vm.network :hostonly, "10.0.33.10"|
|# Share directory between the host and the VM|
|<xsl:output version="1.0" encoding="utf-8" indent="yes" name="xml" />|