Skip to content

Instantly share code, notes, and snippets.

@nmukerje

nmukerje/gist:9bcdd58343c70a3a39f2a31134608360

Created November 8, 2018 20:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nmukerje/9bcdd58343c70a3a39f2a31134608360 to your computer and use it in GitHub Desktop.
Save nmukerje/9bcdd58343c70a3a39f2a31134608360 to your computer and use it in GitHub Desktop.
Download ZIP
EMRFS Consistency TTL
Raw
gistfile1.txt
=== Questions ===
How to automate to delete expired data from EmrFSMetadata
=== Actions ===
step 1: enable TTL on deletionTTL attribute in EmrFSMetadata dynamodb table;
step 2: update emrfs-site.xml on all nodes in the EMR cluster.
Set fs.s3.consistent.metadata.delete.ttl.enabled to true
Set fs.s3.consistent.metadata.delete.ttl.expiration.seconds to 3600 (1h)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment