spark编译调试错误

spark_compile_debug_errors.md

• 运行exmaple中示例错误1:

Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration

VM options启动参数增加:

-Dspark.master=local

参考解决A master URL must be set in your configuration错误

• 运行example示例错误2:

Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Seq

解决方法:

Menu -> File -> Project Structure -> Modules -> spark-examples_2.11 -> Dependencies 添加依赖 jars -> {spark dir}/spark/assembly/target/scala-2.11/jars/

参考http://www.infocool.net/kb/Spark/201612/260785.html

• 编译错误

not found: type SparkFlumeProtocol

参考import Spark source code into intellj, build Error: not found: type SparkFlumeProtocol and EventBatch

nice,Thx