Last active
December 30, 2016 00:59
-
-
Save yaravind/1dcb35bf84f2bddabeb7d583371a9294 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TITLE Launcher - Spark Master, 1 Worker and History Server | |
set SPARK_HOME=C:\aravind\sw\spark-2.0.2-bin-hadoop2.7 | |
:: - Find the IP Address and set it to IP_ADDR env var and reuse it while launching Worker | |
for /f "tokens=1-2 delims=:" %%a in ('ipconfig^|find "IPv4"') do set ip=%%b | |
set IP_ADDR=%ip:~1% | |
echo %IP_ADDR% | |
:: - Start master | |
START "Spark Master" %SPARK_HOME%\bin\spark-class.cmd org.apache.spark.deploy.master.Master | |
ECHO Started Spark MASTER - Web UI on http://localhost:8080 | |
%% Wait for master to complete initialization so that worker can connect to it | |
PING -n 15 127.0.0.1 > NUL | |
:: - Start worker with 1 core and 64mb ram | |
START "Spark Worker" %SPARK_HOME%\bin\spark-class.cmd org.apache.spark.deploy.worker.Worker -c 1 -m 1g -d C:/aravind/sw/spark-worker-dir spark://%IP_ADDR%:7077 | |
ECHO Started Spark WORKER - Web UI on http://localhost:8081 | |
PING -n 5 127.0.0.1 > NUL | |
START "Spark History Server" %SPARK_HOME%\bin\spark-class.cmd org.apache.spark.deploy.history.HistoryServer file:///C:/aravind/sw/spark-shared-eventlog | |
ECHO Started Spark WORKER - Web UI on http://localhost:18080 and REST API on http://localhost:18080/api/v1/applications | |
PING -n 5 127.0.0.1 > NUL | |
:: - Launch WebUI's in defualt browser | |
start chrome http://%IP_ADDR%:8080 | |
start chrome http://%IP_ADDR%:8081 | |
start chrome http://%IP_ADDR%:18080 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I chose to launch the WebUI's in chrome. If you want them to be launched in the default browser remove 'chrome' i.e.
start http://%IP_ADDR%:8080