Skip to content

Instantly share code, notes, and snippets.

@drorata
Created March 14, 2016 14:42
Show Gist options
  • Save drorata/5ec4f63b961b9b4025c5 to your computer and use it in GitHub Desktop.
Save drorata/5ec4f63b961b9b4025c5 to your computer and use it in GitHub Desktop.
Toree, Jupyter and Spark
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Toree on Mac\n",
"\n",
"## Installing\n",
"\n",
"This seems to be easy. First note that my `Jupyter` is installed using `Anaconda` on my user's spaces. As indicated on the project's [GitHub](https://github.com/apache/incubator-toree/blob/master/README.md) the installation is simple:\n",
"\n",
"1. `pip install toree`\n",
"2. `jupyter toree install`\n",
"\n",
"My problem was, that since my Jupyter is local, the second step yielded a permission error. The way around it is:\n",
"\n",
" jupyter toree install --spark_home=/Users/username/Applications/spark --user\n",
" \n",
"Next, I could simply run `jupyter notebook` and start use `Scala/Spark` within `Jupyter`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example --- Compute Pi"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"val NUM_SAMPLES = 10000000"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Pi is roughly 3.1416408\n"
]
}
],
"source": [
"val count = sc.parallelize(1 to NUM_SAMPLES).map{i =>\n",
" val x = Math.random()\n",
" val y = Math.random()\n",
" if (x*x + y*y < 1) 1 else 0\n",
"}.reduce(_ + _)\n",
"println(\"Pi is roughly \" + 4.0 * count / NUM_SAMPLES)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Toree",
"language": "",
"name": "toree"
},
"language_info": {
"name": "scala"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment