Skip to content

Instantly share code, notes, and snippets.

@BrendanThompson

BrendanThompson/session-config.scala Secret

Created September 8, 2021 00:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save BrendanThompson/f8197491b19c192f42658c8936c6778d to your computer and use it in GitHub Desktop.
Save BrendanThompson/f8197491b19c192f42658c8936c6778d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
session-config.scala
val appID = "<appID>"
val secret = "<secret>"
val tenantID = "<tenant-id>"
spark.conf.set("fs.azure.account.auth.type", "OAuth")
spark.conf.set("fs.azure.account.oauth.provider.type", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
spark.conf.set("fs.azure.account.oauth2.client.id", "<appID>")
spark.conf.set("fs.azure.account.oauth2.client.secret", "<secret>")
spark.conf.set("fs.azure.account.oauth2.client.endpoint", "https://login.microsoftonline.com/<tenant-id>/oauth2/token")
spark.conf.set("fs.azure.createRemoteFileSystemDuringInitialization", "true")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment