Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
AWS SSM SSH Proxy Command
#!/usr/bin/env sh
set -eu -o pipefail
######## Usage #################################################################
#
# #1 Install the AWS CLI
# https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
#
# #2 Install the Session Manager Plugin for the AWS CLI
# https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html
#
# #3 Install ProxyCommand
# - Move this script to ~/.ssh/aws-ssm-ec2-proxy-command.sh
# - Make it executable (chmod +x ~/.ssh/aws-ssm-ec2-proxy-command.sh)
#
# #4 Setup SSH Config
# - Add foolowing entry to your ~/.ssh/config
#
# host i-* mi-*
# ProxyCommand ~/.ssh/aws-ssm-ec2-proxy-command.sh %h %r %p
#
# #5 Ensure SSM Permissions of Target Instance Profile
#
# https://docs.aws.amazon.com/systems-manager/latest/userguide/setup-instance-profile.html
#
# #6 Ensure latest SSM Agent on Target Instance
#
# Then SSM agent is preinstalled on all amazon linux AMIs, however may needs to be updated,
#
# Run following command to update SSM agent on target instance
#
# aws ssm send-command \
# --document-name "AWS-UpdateSSMAgent" \
# --document-version '$LATEST' \
# --instance-ids "${ec2_instance_id}"
#
# #7 Finally connect to ec2 instance
#
# export AWS_PROFILE='default'
# ssh ec2-user@i-xxxxxxxxxxxxxxxx
#
# or just
#
# AWS_PROFILE='default' ssh ec2-user@i-xxxxxxxxxxxxxxxx
#
################################################################################
ec2_instance_id="$1"
ssh_user="${2}"
ssh_port="${3}"
ssh_public_key_path="${HOME}/.ssh/id_rsa.pub"
ssh_authorized_key_timeout=10
# Try to get an public ssh key from 'ssh agent'
ssh_public_key="$(keys="$(ssh-add -L | head -1)" && echo "$keys" || true)"
if [ -n "$ssh_public_key" ]; then
ssh_public_key_source='ssh agent'
else
# Try read public ssh key from '${ssh_public_key_path}'
ssh_public_key="$([[ -e "${ssh_public_key_path}" ]] && cat "${ssh_public_key_path}")"
if [ -n "$ssh_public_key" ]; then
ssh_public_key_source="${ssh_public_key_path}"
fi
fi
if [ -z "$ssh_public_key" ]; then
echo "No ssh key present in ssh agent nor at ${ssh_public_key_path}"
exit 1
fi
echo "Temporary add your public ssh key from '$ssh_public_key_source' to authorized_keys on target instance ${ec2_instance_id}"
aws ssm send-command \
--instance-ids "${ec2_instance_id}" \
--document-name 'AWS-RunShellScript' \
--parameters commands="\"
cd ~${ssh_user}/.ssh || exit 1
grep -F '${ssh_public_key}' authorized_keys || echo '${ssh_public_key} ssm-session' >> authorized_keys
sleep ${ssh_authorized_key_timeout}
grep -v -F '${ssh_public_key}' authorized_keys > .tmp.authorized_keys
mv .tmp.authorized_keys authorized_keys
\"" \
--comment "grant ssh access for ${ssh_authorized_key_timeout} seconds"
# Start SSM SSH session
aws ssm start-session \
--target "${ec2_instance_id}" \
--document-name 'AWS-StartSSHSession' \
--parameters "portNumber=${ssh_port}"
@candrews

This comment has been minimized.

Copy link

candrews commented Oct 8, 2019

Instead of reading the public key from file, how about using ssh-add -L to get it instead? That way, this script will work for users who use smart cards and ssh agents, for example, instead of file-based keys.

@qoomon

This comment has been minimized.

Copy link
Owner Author

qoomon commented Oct 9, 2019

That's a nice idea. I will modify the script.

@qoomon

This comment has been minimized.

Copy link
Owner Author

qoomon commented Oct 9, 2019

@candrews which key I should select if there are multiple keys returned by ssh-add -L? the first one? or is there a way to select a specific one?

my current idea would look like this

ssh_public_key="$((keys="$(ssh-add -L)" && echo $keys | head -1) || cat "${ssh_public_key_path}")"
@candrews

This comment has been minimized.

Copy link

candrews commented Oct 9, 2019

I suggest selecting the first one as I can't think of any way to decide which (if any) is better than any other.

@qoomon

This comment has been minimized.

Copy link
Owner Author

qoomon commented Oct 9, 2019

Done. WDYT

@candrews

This comment has been minimized.

Copy link

candrews commented Oct 10, 2019

Done. WDYT

Awesome! Thank you!

@at-bachhuynh

This comment has been minimized.

Copy link

at-bachhuynh commented Nov 1, 2019

I got this error @qoomon

ssh ec2-user@i-06cc99dca3e71e40d
ssh_exchange_identification: Connection closed by remote host

I follow your steps but got above error.
How can I fix it? Thank you!

Here is log (set -x)

+ ec2_instance_id=i-06cc99dca3e71e40d
+ ssh_user=ec2-user
+ ssh_port=22
+ ssh_public_key_path=/Users/bach.huynh/.ssh/id_rsa.pub
+ ssh_public_key_timeout=10
+++ ssh-add -L
++ keys='The agent has no identities.'
+ ssh_public_key=
ssh_exchange_identification: Connection closed by remote host```
@qoomon

This comment has been minimized.

Copy link
Owner Author

qoomon commented Nov 1, 2019

Thank for reporting, I've found the bug.
following line failed because of set -o pipefail
ssh_public_key="$(keys="$(ssh-add -L)" && echo $keys | head -1)"
I've changed that to
ssh_public_key="$(keys="$(ssh-add -L | head -1)" && echo "$keys" || true)"
Should work now

@at-bachhuynh

This comment has been minimized.

Copy link

at-bachhuynh commented Nov 2, 2019

Goodmorning @qoomon (from my timezone :-) )
I try to fix by the new line:
First, I make a test by start-session to ensure my instance running

aws ssm start-session --target i-06b0e5a301a208ce7
Starting session with SessionId: botocore-session-1572660806-0d9845789f2710780
sh-4.2$ sudo su
[root@ip-192-168-40-201 /]# exit
exit
sh-4.2$ exit
exit
Exiting session with sessionId: botocore-session-1572660806-0d9845789f2710780.

It's running,
Now, test your script:

ssh ec2-user@i-06b0e5a301a208ce7
+ ec2_instance_id=i-06b0e5a301a208ce7
+ ssh_user=ec2-user
+ ssh_port=22
+ ssh_public_key_path=/Users/bach.huynh/.ssh/id_rsa.pub
+ ssh_public_key_timeout=10
+++ ssh-add -L
+++ head -1
++ keys='ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4ESNGTDO7NOUd4wljzm44XgmJ9s2lIRCzhRPFggn+cBJcillVCrXQ/IK+l67dCymAkevhjG5TOl/p+yKBJrY/obRNslTNGZHAmAAr3ml4QVqVsUNmOvUiVpKlhsKKws/ByGX0Q8GP7oNEVnPUl4mCpXopGwjPCO6yE9MVD0RMAlWncOgKGVVumvD5KQ9DhLFqNxjxZBIa79hiwlygSxrOw16MGUzFR1eVNl3zo+NAu2M8tN5nfap3TG9mQThFXahXWF1WzfUTMLlZOifI79+1haacig5d8roeJCvwoEkl2t8REt7fCdHnE3wa0LsT5gnXSyTtzV+aPQ2tWVQsnr+P '
++ echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4ESNGTDO7NOUd4wljzm44XgmJ9s2lIRCzhRPFggn+cBJcillVCrXQ/IK+l67dCymAkevhjG5TOl/p+yKBJrY/obRNslTNGZHAmAAr3ml4QVqVsUNmOvUiVpKlhsKKws/ByGX0Q8GP7oNEVnPUl4mCpXopGwjPCO6yE9MVD0RMAlWncOgKGVVumvD5KQ9DhLFqNxjxZBIa79hiwlygSxrOw16MGUzFR1eVNl3zo+NAu2M8tN5nfap3TG9mQThFXahXWF1WzfUTMLlZOifI79+1haacig5d8roeJCvwoEkl2t8REt7fCdHnE3wa0LsT5gnXSyTtzV+aPQ2tWVQsnr+P '
+ ssh_public_key='ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4ESNGTDO7NOUd4wljzm44XgmJ9s2lIRCzhRPFggn+cBJcillVCrXQ/IK+l67dCymAkevhjG5TOl/p+yKBJrY/obRNslTNGZHAmAAr3ml4QVqVsUNmOvUiVpKlhsKKws/ByGX0Q8GP7oNEVnPUl4mCpXopGwjPCO6yE9MVD0RMAlWncOgKGVVumvD5KQ9DhLFqNxjxZBIa79hiwlygSxrOw16MGUzFR1eVNl3zo+NAu2M8tN5nfap3TG9mQThFXahXWF1WzfUTMLlZOifI79+1haacig5d8roeJCvwoEkl2t8REt7fCdHnE3wa0LsT5gnXSyTtzV+aPQ2tWVQsnr+P '
+ '[' -n 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4ESNGTDO7NOUd4wljzm44XgmJ9s2lIRCzhRPFggn+cBJcillVCrXQ/IK+l67dCymAkevhjG5TOl/p+yKBJrY/obRNslTNGZHAmAAr3ml4QVqVsUNmOvUiVpKlhsKKws/ByGX0Q8GP7oNEVnPUl4mCpXopGwjPCO6yE9MVD0RMAlWncOgKGVVumvD5KQ9DhLFqNxjxZBIa79hiwlygSxrOw16MGUzFR1eVNl3zo+NAu2M8tN5nfap3TG9mQThFXahXWF1WzfUTMLlZOifI79+1haacig5d8roeJCvwoEkl2t8REt7fCdHnE3wa0LsT5gnXSyTtzV+aPQ2tWVQsnr+P ' ']'
+ ssh_public_key_source='ssh agent'
+ echo 'Temporary add your public ssh key from '\''ssh agent'\'' to authorized_keys on target instance i-06b0e5a301a208ce7'
+ aws ssm send-command --instance-ids i-06b0e5a301a208ce7 --document-name AWS-RunShellScript --parameters 'commands="
    cd ~ec2-user/.ssh || exit 1
    grep -F '\''ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4ESNGTDO7NOUd4wljzm44XgmJ9s2lIRCzhRPFggn+cBJcillVCrXQ/IK+l67dCymAkevhjG5TOl/p+yKBJrY/obRNslTNGZHAmAAr3ml4QVqVsUNmOvUiVpKlhsKKws/ByGX0Q8GP7oNEVnPUl4mCpXopGwjPCO6yE9MVD0RMAlWncOgKGVVumvD5KQ9DhLFqNxjxZBIa79hiwlygSxrOw16MGUzFR1eVNl3zo+NAu2M8tN5nfap3TG9mQThFXahXWF1WzfUTMLlZOifI79+1haacig5d8roeJCvwoEkl2t8REt7fCdHnE3wa0LsT5gnXSyTtzV+aPQ2tWVQsnr+P '\'' authorized_keys || echo '\''ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4ESNGTDO7NOUd4wljzm44XgmJ9s2lIRCzhRPFggn+cBJcillVCrXQ/IK+l67dCymAkevhjG5TOl/p+yKBJrY/obRNslTNGZHAmAAr3ml4QVqVsUNmOvUiVpKlhsKKws/ByGX0Q8GP7oNEVnPUl4mCpXopGwjPCO6yE9MVD0RMAlWncOgKGVVumvD5KQ9DhLFqNxjxZBIa79hiwlygSxrOw16MGUzFR1eVNl3zo+NAu2M8tN5nfap3TG9mQThFXahXWF1WzfUTMLlZOifI79+1haacig5d8roeJCvwoEkl2t8REt7fCdHnE3wa0LsT5gnXSyTtzV+aPQ2tWVQsnr+P  ssm-session'\'' >> authorized_keys
    sleep 10
    grep -v -F '\''ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4ESNGTDO7NOUd4wljzm44XgmJ9s2lIRCzhRPFggn+cBJcillVCrXQ/IK+l67dCymAkevhjG5TOl/p+yKBJrY/obRNslTNGZHAmAAr3ml4QVqVsUNmOvUiVpKlhsKKws/ByGX0Q8GP7oNEVnPUl4mCpXopGwjPCO6yE9MVD0RMAlWncOgKGVVumvD5KQ9DhLFqNxjxZBIa79hiwlygSxrOw16MGUzFR1eVNl3zo+NAu2M8tN5nfap3TG9mQThFXahXWF1WzfUTMLlZOifI79+1haacig5d8roeJCvwoEkl2t8REt7fCdHnE3wa0LsT5gnXSyTtzV+aPQ2tWVQsnr+P '\'' authorized_keys > .tmp.authorized_keys
    mv .tmp.authorized_keys authorized_keys
  "' --comment 'grant ssh access for 10 seconds'
+ aws ssm start-session --target i-06b0e5a301a208ce7 --document-name AWS-StartSSHSession --parameters portNumber=22

Stuck here, I have to Ctrl+C to exit

^C

Command '['session-manager-plugin', '{"SessionId": "botocore-session-1572660806-040cef5746ca4c2c8", "TokenValue": "AAEAAUsxWBg0Gc4B3DPxQ3/ys7izmtUcWjxBhmufskFCAIHZAAAAAF285+11reFgn5MCPuYU5JKTw/IB2JtyQ59rhuWd5b6q2VDTJ8YOqg+egPwhHX4/Z2YixFPhcYazL8ItJ48WkT24MVqJjip4nG0BxvqJ/V4RFYrVcRIu7WNz9StAeqhcPGvlIOvOUO2hlfvWewGem+qHdyZmfP5Iarj8WfNoPdNO8Tl+ekXCMW84WRUQj1lMRLDSaXcOTyBXtNM8Q5C6GJtIIs9InezuGmv28r0iExydka9D0F0ttc9UBLWCE5xixVYIa+pCnO9NvflmALgYOpBH5r64p+M3/rIMCEWPEDueh7TWpsMN+M6OpoVqjx+h6fzCBuJOes6dmOezVdOzwK4C2GRDr4r3DCM0rl+pgZrxo+hHlmYux3hmbDeHG7qP8g+MwCc=", "StreamUrl": "wss://ssmmessages.ap-northeast-1.amazonaws.com/v1/data-channel/botocore-session-1572660806-040cef5746ca4c2c8?role=publish_subscribe", "ResponseMetadata": {"RetryAttempts": 0, "HTTPStatusCode": 200, "RequestId": "e6a8d2f9-53c9-4f4f-aa37-a32c931813d8", "HTTPHeaders": {"x-amzn-requestid": "e6a8d2f9-53c9-4f4f-aa37-a32c931813d8", "date": "Sat, 02 Nov 2019 02:20:29 GMT", "content-length": "667", "content-type": "application/x-amz-json-1.1"}}}', 'ap-northeast-1', 'StartSession', 'be-quality-dev', '{"DocumentName": "AWS-StartSSHSession", "Target": "i-06b0e5a301a208ce7", "Parameters": {"portNumber": ["22"]}}', u'https://ssm.ap-northeast-1.amazonaws.com']' returned non-zero exit status -13
@at-bachhuynh

This comment has been minimized.

Copy link

at-bachhuynh commented Nov 2, 2019

hi @qoomon
I find out that we need to run update SSM first to make sure SSM agent is updated with latest version, I put this code to your script from line 41

# Update SSM agent
aws ssm send-command --document-name "AWS-UpdateSSMAgent" \
--document-version "\$LATEST" \
--targets "Key=instanceids,Values=${ec2_instance_id}" \
--parameters '{"version":[""],"allowDowngrade":["false"]}' \
--timeout-seconds 600 \
--max-concurrency "50" \
--max-errors "0" \```

And now on, it's working but sometime it need to be run second time to ssh successfully
@qoomon

This comment has been minimized.

Copy link
Owner Author

qoomon commented Nov 2, 2019

@at-bachhuynh yes it's right old ssm agent versions do not work for reasons #6 Ensure latest SSM Agent on Target Instance
I think your command is quite handy to do that however it think it shouldn't be part auf tohe proxy command, this is something you should ensure by a good base image. I think I will add you snippet to the readme section #6 Ensure latest SSM Agent on Target Instance

However I have some notes regarding your snippet.

  • why --max-concurrency needs to be set at all or set to such a high value as 50 ?
  • does it make sense to set --timeout-seconds 600 ?
  • --parameters '{"version":[""],"allowDowngrade":["false"]}' \ , is obsolete, cause these are default values.
  • --max-errors "0" is obsolete, cause 0 is the default value

I think following command should be sufficient

aws ssm send-command \
--document-name "AWS-UpdateSSMAgent" \
--document-version '$LATEST' \
--instance-ids "${ec2_instance_id}"

WDYT?

@at-bachhuynh

This comment has been minimized.

Copy link

at-bachhuynh commented Nov 4, 2019

Hi @qoomon
After many time connect ssh by ssm,
I think we should put my script` into your script to ensure SSM always update first.
The later access is still fast, nothing to worry about speed or timeout.

@qoomon

This comment has been minimized.

Copy link
Owner Author

qoomon commented Nov 4, 2019

@at-bachhuynh I have concerns about changing instance state by connecting to an instance via SSH, cause it's an unexpected side effect, I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.