Skip to content

Instantly share code, notes, and snippets.

@nmukerje
Last active February 8, 2018 00:48
Show Gist options
  • Save nmukerje/d66b33aa1c6f2a437ccba2483ea0992f to your computer and use it in GitHub Desktop.
Save nmukerje/d66b33aa1c6f2a437ccba2483ea0992f to your computer and use it in GitHub Desktop.
SSH to EMR Master
import paramiko
## get keypair from S3 or application host
k = paramiko.RSAKey.from_private_key_file("<keypair.pem> file")
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
print "connecting"
c.connect( hostname = "<emr master node>", username = "hadoop", pkey = k )
print "connected"
command1='nohup sqoop import -D mapred.job.name=SqoopTest121 --connect jdbc:postgresql://db.rds.amazonaws.com:5432/apostgres --username user --table random_data --m 10 --password XXXX --split-by id >> /tmp/logs/sqoop/SqoopTest121.log 2>&1 &'
print "Executing {}".format( command )
stdin , stdout, stderr = c.exec_command(command)
c.close()
## Save job state in DynamoDB.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment