Skip to content

Instantly share code, notes, and snippets.

@peterwwillis
Last active December 7, 2023 20:52
Show Gist options
  • Star 7 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save peterwwillis/b447ab446a2052f3a6b566928551da56 to your computer and use it in GitHub Desktop.
Save peterwwillis/b447ab446a2052f3a6b566928551da56 to your computer and use it in GitHub Desktop.
This gist will track the websites that suggest unnecessarily-insecure default ssh-keygen arguments, in an attempt to have their authors update their arguments to safer defaults.
#!/usr/bin/env bash
declare -A LIST_OF_GUIDES
LIST_OF_GUIDES=(
["https://confluence.atlassian.com/bitbucketserver/creating-ssh-keys-776639788.html"]="Atlassian"
["https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/"]="GitHub"
["https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2"]="DigitalOcean"
["https://www.ssh.com/ssh/keygen/"]="info.us@ssh.com"
["https://git-scm.com/book/en/v2/Git-on-the-Server-Generating-Your-SSH-Public-Key"]="Git SCM"
["https://www.vultr.com/docs/how-do-i-generate-ssh-keys"]="Vultr"
["https://docs.joyent.com/public-cloud/getting-started/ssh-keys/generating-an-ssh-key-manually/manually-generating-your-ssh-key-in-mac-os-x"]="support@joyent.com"
["https://help.ubuntu.com/community/SSH/OpenSSH/Keys"]="Ubuntu"
["https://docs.gitlab.com/ee/ssh/README.html"]="GitLab"
["https://jdblischak.github.io/2014-09-18-chicago/novice/git/05-sshkeys.html"]="admin@software-carpentry.org"
["https://www.scaleway.com/docs/configure-new-ssh-key/"]="Scaleway"
["https://help.packet.net/quick-start/generate-ssh-keys"]="help@packet.net"
["https://inchoo.net/dev-talk/how-to-generate-ssh-keys-for-git-authorization/"]="web@inchoo.net"
["https://www.siteground.com/kb/generate_ssh_key_in_linux/"]="sales@siteground.com"
["https://www.howtoforge.com/linux-basics-how-to-install-ssh-keys-on-the-shell"]="HowtoForge"
["https://kb.iu.edu/d/aews"]="UITS"
["http://www.oracle.com/webfolder/technetwork/tutorials/obe/cloud/javaservice/JCS/JCS_SSH/create_sshkey.html"]="Oracle"
["https://mediatemple.net/community/products/dv/204644740/using-ssh-keys-on-your-server"]="MediaTemple"
["https://www.debian.org/devel/passwordlessssh"]="Debian"
["https://wiki.archlinux.org/index.php/SSH_keys"]="Arch Linux"
["https://docs.rightscale.com/faq/How_Do_I_Generate_My_Own_SSH_Key_Pair.html"]="RightScale"
["https://www.w3docs.com/snippets/git/how-to-generate-ssh-key-for-git.html"]="W3Docs"
["https://www.linode.com/docs/security/authentication/use-public-key-authentication-with-ssh/"]="support@linode.com"
["https://www.fullstackpython.com/blog/ssh-keys-ubuntu-linux.html"]="mattmakai"
["https://support.rackspace.com/how-to/generating-rsa-keys-with-ssh-puttygen/"]="RackSpace"
["https://docs.microsoft.com/en-us/azure/virtual-machines/linux/mac-create-ssh-keys"]="Microsoft Azure"
["https://www.keycdn.com/support/create-ssh-key/"]="KeyCDN"
["https://devcenter.heroku.com/articles/keys"]="Heroku"
)
declare -A SENT_TO
SENT_TO=(
["Heroku"]="article feedback"
["KeyCDN"]="feature request"
["mattmakai"]="https://github.com/mattmakai/fullstackpython.com/pull/182"
["support@linode.com"]="docs contributing submission"
["RightScale"]="feedback form"
["Arch Linux"]="https://bugs.archlinux.org/task/59522"
["Debian"]="https://salsa.debian.org/webmaster-team/webwml/merge_requests/15"
["MediaTemple"]="feedback form"
["UITS"]="feedback form"
["HowtoForge"]="feedback form"
["sales@siteground.com"]="e-mail"
["web@inchoo.net"]="feedback form"
["help@packet.net"]="e-mail"
["Scaleway"]="https://cloud.scaleway.com/#/tickets/EXOY-8867-NOCP"
["admin@software-carpentry.org"]="https://github.com/jdblischak/2014-09-18-chicago/pull/3"
["GitLab"]="https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6796"
["Vultr"]="docs contributing submission"
["Git SCM"]="https://github.com/progit/progit2/pull/1099"
["info.us@ssh.com"]="e-mail"
["RackSpace"]="https://community.rackspace.com/feedback/f/how-to-api-documentation/8816/insecure-use-of-ssh-keygen"
["DigitalOcean"]="https://cloudsupport.digitalocean.com/s/case/5001N00000dihowQAA"
["GitHub"]="feedback form"
["Atlassian"]="feedback form"
["W3Docs"]="facebook,e-mail"
)
LETTER_TO_THE_EDITOR=" echo \"
To: \$TO_EMAIL
From: \$FROM_EMAIL
Subject: Insecure use of ssh-keygen
Hello, this is \$MY_NAME. I found your guide on generating SSH keys (\$URL) after looking at the top 20 Google results for 'how do I generate an ssh key', and it looks like your default suggestions for using ssh-keygen are insecure.
I highly recommend you add this to your guide's ssh-keygen arguments:
ssh-keygen -o
(Note: The -o option was added in 2014; if this command fails for you, simply remove the -o option)
This default may introduce compatibility issues for users of older computers.
In that case, please simply provide the following paragraphs in your SSH guide:
> The default password encryption used for ssh keys is MD5(plaintext | IV).
> The security of the MD5 hash has been severely compromised, and such passwords
> are easily cracked using modern computers and techniques.
> Since 2014, ssh-keygen has provided a new key format with greatly improved
> password encryption. To use this new format, simply add the '-o' option to
> ssh-keygen. No further steps are required. Keys using this format will be
> incompatible with versions of OpenSSH older than 6.5 (released in 2014).
> If you will be using your key with versions of SSH older than this, it is
> suggested NOT to set a password on your key. The password will be easily
> decrypted by any modern attacker and then used to attack other accounts.
The problem is described in this blog post: https://latacora.singles/2018/08/03/the-default-openssh.html
More info can be found on this Hacker News thread: https://news.ycombinator.com/item?id=17682999
By making these changes to your guide, you will be making the internet safer. If the keys used to access servers are secure by default, then both the servers, and all the users who visit websites on those servers, will be more secure.
---
Here is some more background on the problem.
From the ssh-keygen manual:
-o Causes ssh-keygen to save private keys using the new OpenSSH for‐
mat rather than the more compatible PEM format. The new format
has increased resistance to brute-force password cracking but is
not supported by versions of OpenSSH prior to 6.5. Ed25519 keys
always use the new private key format.
By default, ssh-keygen will try to password-protect the ssh key. But the default encoding for the password is actually very easy to crack. By using the -o argument you can change to a newer password encoding which is much more secure.
Using the default key encryption is actually worse than not providing a password at all. The password is so easy to crack that it can then be reused to attack other accounts that use this password. (See the above articles for more details)
The only downside to using -o is it may not work for versions of ssh-keygen older than 2014. You can therefore suggest the -o option, and suggest that if it does not work, that the user remove the -o option.
Additional recommendations:
- Your users can upgrade their existing keys with improved password security with this command:
ssh-keygen -p -o -f (oldfile)
- If your guide uses the PuTTYgen tool, please suggest AT LEAST 2048 for the number of bits in a generated key. PuTTYgen's default of 1024 is far too small to be secure.
Thank you very much for your consideration and help in this matter.
Please feel free to contact me if you have any questions.
\"
"
#[ -n "$TO_EMAIL" ] || read -p "Enter the recipient e-mail address: " TO_EMAIL
[ -n "$FROM_EMAIL" ] || read -p "Enter your e-mail address: " FROM_EMAIL
[ -n "$MY_NAME" ] || read -p "Enter your name: " MY_NAME
[ -d "letters" ] || mkdir letters
for URL in "${!LIST_OF_GUIDES[@]}" ; do
#echo ""
#read -p "Press Enter to generate the next letter..."
TO_EMAIL="${LIST_OF_GUIDES[$URL]}"
LETTER=`eval "$LETTER_TO_THE_EDITOR"`
echo "$LETTER" > letters/"$TO_EMAIL".txt
done
@peterwwillis
Copy link
Author

peterwwillis commented Sep 17, 2018

I added the following to a DigitalOcean Q&A thread to increase awareness, posted here for posterity.


tl;dr if you didn't use the ed25519 SSH key type or didn't use the "-o" option to ssh-keygen, your SSH keys are insecure

If you never set a password on your ssh keys, you can ignore all of this. But if you did set a password on your SSH key, and you created your ssh key with the RSA, DSA, or ECDSA key types (RSA is the default), your private key is probably vulnerable to password cracking.

All of this specifically applies to OpenSSH.

What's going on here?

In January of 2014, ssh version 6.5 was released. One of the new features was a new, optional format for creating private keys. The old format was starting to look like it wouldn't stand up to brute-force password attacks in the future, so they added a newer, stronger format that would prevent brute forcing.

But they didn't make this new format the default. It was supposed to become the default, but it never did. The problem was compatibility: if you made a key with the new format, and then tried to use it on an old version of SSH, it wouldn't work, and this could cause some confusion. So they kept the old format the default for compatibility reasons.

The end result is we've been making SSH keys for years with password encryption that's no longer very secure against brute-force attacks. This website has the technical details: https://latacora.singles/2018/08/03/the-default-openssh.html

But it's been four years, and most people are now using new versions of SSH that use the new format. Go ahead and run ssh -V on all your servers. If it's version 6.5 or newer, your computers will work with the new key format.

First, some brief terminology

  • private key: a secret chunk of data that SSH uses to create an encrypted connection between you and a server. It's usually called "id_rsa", "id_dsa", "id_ecdsa", or "id_ed2559".
  • public key: a public or non-secret piece of data that an SSH server uses to allow you to log-in to it. It's usually got the same name as the private key but with ".pub" added at the end.

How do I make my private key secure?

You have two options: 1) re-encrypt your existing ssh private key, or 2) create a new ed25519 type key.

  1. Just run this command: ssh-keygen -p -o -f (oldfile)

This will re-encrypt your existing private key file. Put in the old password, then the same password again as the new password (or better yet, come up with a different password, in case your key got stolen before).

  1. The ed25519 key type automatically uses the newer key format, because old versions of ssh don't support these keys anyway. Use these if none of your servers are older than SSH version 6.5.

What if I'm using a version of SSH older than 6.5 on some of my servers?

If those servers only have your public key, don't sweat it. Only the private key format is changed, so those servers won't notice.

If those servers have your private key, your best bet is to actually run the following command, and for the new password, don't enter any password at all:
ssh-keygen -p -f (oldfile)

The reason why you shouldn't use a password at all (on old versions of SSH) is that if someone gets the private key, they can probably crack the password. If they get the password, then they know what kind of passwords you use, and can attack other accounts you might have. So it's better to just remove the password entirely than give an attacker one of your passwords.

Why do I need to password-protect my SSH keys anyway?

Your SSH key is like a password that lets you log in to your servers. By keeping these as files on a disk, it's the same as writing a password to the disk as a plain text file. If somebody gets access to your hard disk, backups, etc, anybody can read it and use it to log into your servers. Password protection means the key file is protected, so if anyone gets the file, nobody can use the keys except you.

Normally you would have to type this password in every time you connected to your server, which can be a pain. But by using an ssh-agent program, you only have to enter this password the first time you start up your computer. You can find a guide (on this page)[http://mah.everybody.org/docs/ssh] and a detailed explanation of (how SSH agents work here)[http://www.unixwiz.net/techtips/ssh-agent-forwarding.html]

Is this for real?

Yes. I have contacted DigitalOcean and asked them to update their official documentation. Some other sites have started update their documentation, like the Arch Linux wiki, GitLab documentation, the Git SCM documentation, and a few others.

You can get more details from this page, including using the ed25519 key type, and adding the -a option to ssh-keygen to increase the level of password protection on your SSH key.

@peterwwillis
Copy link
Author

peterwwillis commented Sep 17, 2018

It's been a while since I originally reached out to these sites to see if they would update their documentation.

Some organizations and individuals have been really supportive of the added documentation, and worked quickly to update their docs. These were mostly sites where I could e-mail the product owner directly, or if it was a very small company or project, or where I could submit a pull-request directly against the documentation.

Some (mostly corporations) were hesitant to include the new documentation, and replied that they'd discuss it internally. These sites do not appear to have updated their documentation. This includes GitHub, one of the first search results for ssh key generation.

One just refused outright. I'll name them below because I think their users are at risk, and the reasons for their refusal are dumbfounding.

Indiana University's University Information Technology Services refused to modify their documentation. The reasoning I was given were:

  1. "Because our systems are using an older version of SSH that doesn't support the -o option"
  2. "because the content is primarily meant for students and beginner-level Linux users" and "To avoid confusing students"

The first may be confusion on their part, because the server shouldn't need to support the new private key format. Only the client needs to. If you use RSA keys with the -o option, even old servers should support it.

The second is just silly. This is an educational institution, saying they will not educate their students, because to do so might confuse them.

If you go to the UITS ssh key page, it states now that beginning in October 2018, their services will require two-factor auth - unless an individual signs a pledge and agrees to password protect their SSH key. But they still don't tell those users that the default password protection method is bunk. They explicitly mention multiple times that all users should password protect their key, but refuse to tell their users how to do this safely.

What I take from these examples is that organizations prefer the perception of security to the reality. I think this is a good lesson in security design. If you design a security feature in a way that it can be implemented poorly without raising suspicion, people will continue to implement it poorly. So the ability to flag bad security practices may be even more important than having security features at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment