spark

Install Scala 

brew install scala 



Set SCALA_HOME 

export SCALA_HOME=/usr/local/bin/scala  
export PATH=$PATH:$SCALA_HOME/bin 

  
Download Spark from https://spark.apache.org/downloads.html  

tar -xvzf spark-1.1.1.tar 
cd spark-1.1.1  


Build and Install Apache Spark 

sbt/sbt clean assembly 



Fire up the Spark 

For the Scala shell: 
./bin/spark-shell 

For the Python shell: 
./bin/pyspark 



Run Examples 

Calculate Pi: 

./bin/run-example org.apache.spark.examples.SparkPi 

MLlib Correlations example: 

./bin/run-example org.apache.spark.examples.mllib.Correlations 

MLlib Linear Regression example: 

./bin/spark-submit 
--class org.apache.spark.examples.mllib.LinearRegression 
examples/target/scala-*/spark-*.jar data/mllib/sample_linear_regression_data.txt