This package of shell scripts automates the install and configuration of EMR with Hue, Presto, TLS and SAML.
- The main script uses AWS CLI to install EMR, Hue, and Presto. It drives the other 4 scripts
emr-install-krb-presto-tls.sh
- The actions needed to configure Presto, Kerberos and TLS are in the first bootstrap script
presto-kerberos-tls.sh
- The 2nd bootstrap script configures Hue for SAML authentication
hue_saml_config.sh
- The first step script adjusts the password in presto-env.sh
presto-cli-kerberos_fix.sh
- The 2nd step script makes the final hue.ini and presto config changes
hue_impersonation_presto.sh
A log and a configuration bucket variable are defined at the top of the main install script. Those can be changed to point to site or account specific buckets as needed.
Verify Presto connectivity (presto-cli sources /etc/presto/conf/presto-env.sh):
export PRESTO_PAGER=''
presto-cli --catalog hive --server https://$(hostname -f):8446/ <<< 'show schemas;'
presto-cli --server https://$(hostname -f):8446 --catalog hive --krb5-principal hadoop/$(hostname -f)@EC2.INTERNAL --krb5-keytab-path /etc/hadoop.keytab <<< 'show schemas;'
Verify Hive/HS2/Beeline connectivity:
trustpw=$(awk '/tstore-pass/{print $2}' /etc/presto/conf/presto-env.sh)
beeline -u "jdbc:hive2://$(hostname -f):10000/default;principal=hive/$(hostname -f)@EC2.INTERNAL;auth=kerberos;ssl=true;sslTrustStore=/usr/share/aws/emr/security/conf/truststore.jks;trustStorePassword=$trustpw" --silent <<< 'show databases;'
Verify HUE HTTPS:
openssl s_client -connect $(hostname -f):8888
Download HUE SAML meta-data for upload to https://samltest.id/:
curl -k -L https://$(hostname -f):8888/saml2/metadata -o hue-saml-meta-data.xml
Hi John,
I would like to Kerberised Presto and appreciate if I can use your hard work to achieve it.
Please let me know where can I find these files:
emr-install-krb-presto-tls.sh
presto-kerberos-tls.sh
hue_saml_config.sh
presto-cli-kerberos_fix.sh
hue_impersonation_presto.sh
Thanks in advance for your help in this matter.
Thanks