Skip to content

Instantly share code, notes, and snippets.

@priyanlc

priyanlc/bucket_access.py

Last active April 30, 2020 20:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save priyanlc/330676d08c9098b8294bed7b61bad348 to your computer and use it in GitHub Desktop.
Save priyanlc/330676d08c9098b8294bed7b61bad348 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
bucket_access.py
ACCESS_KEY = <>
SECRET_KEY = <>
AWS_BUCKET_NAME = "s3a://sample-datasets-for-blogs-pl"
MOUNT_NAME= "blogs_pl"
sc._jsc.hadoopConfiguration().set("fs.s3n.awsAccessKeyId", ACCESS_KEY)
sc._jsc.hadoopConfiguration().set("fs.s3n.awsSecretAccessKey", SECRET_KEY)
dbutils.fs.unmount(f"/mnt/{MOUNT_NAME}")
dbutils.fs.mount(AWS_BUCKET_NAME, f"/mnt/{MOUNT_NAME}")
display(dbutils.fs.ls(f"/mnt/{MOUNT_NAME}"))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment