Skip to content

Instantly share code, notes, and snippets.

@hsuyuming

hsuyuming/spark-connector.scala

Created July 2, 2019 18:40
Show Gist options
  • Save hsuyuming/58b9d8fd4c7f9f8061c7174796f715c8 to your computer and use it in GitHub Desktop.
Save hsuyuming/58b9d8fd4c7f9f8061c7174796f715c8 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark-connector.scala
import net.snowflake.spark.snowflake.Utils.SNOWFLAKE_SOURCE_NAME
import org.apache.spark.sql.SparkSession
val sparkSession = SparkSession.builder.master("local[*]").appName("spark session example").getOrCreate()
var sfOptions = Map ("sfUrl"->"<host>","sfAccount"->"<account>","sfUser"->"SPARK","sfPassword"->"<pwd>","sfDatabase"->"CREATE_STAGE_ONLY","sfRole"->"SPARK","sfSchema"->"TEST")
val df = sparkSession.read.format(SNOWFLAKE_SOURCE_NAME).options(sfOptions).option("query","SELECT * FROM SPARK_PERMISSION_TEST.PUBLIC.CUSTOMER;").load()
df.show
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment