Skip to content

Instantly share code, notes, and snippets.

@anirtek
Created July 30, 2019 21:36
Show Gist options
  • Save anirtek/1df57a761161a959cd16b6ccd43206dd to your computer and use it in GitHub Desktop.
Save anirtek/1df57a761161a959cd16b6ccd43206dd to your computer and use it in GitHub Desktop.
Spark code for put object in S3
val spark = SparkSession
.builder()
.appName("SparkScalaApp")
.master("local")
.getOrCreate()
val df = spark.read.csv("/Users/home/Dataset/webLog.csv")
df.createOrReplaceTempView("view")
val result = spark.sql(
"""SELECT Staus, COUNT(*) as Count
|FROM view
|WHERE Staus is 200
|GROUP BY Staus""".stripMargin)
.write
.saveAsTable("logtable")
result.show()
spark.stop()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment