How to manage multiple SSH key pairs

Some sites I interact with use SSH keys for access instead of passwords. As with passwords, I try to make a conscious decision about passphrases, and when to reuse or make new keys.

To manage these keys, I use a combination of command-line options, configuration settings, and passphrase caching agents.

Why use different key pairs?

I currently have about a half dozen places where I use SSH keys on a regular basis and several other less frequently accessed locations. In particular, I use different key pairs for:

• Each of my consulting clients.
• Lab or testing environments.
• Training classrooms and similar environments that use shared keys.
• Networks I manage where the public key is loaded into an identity management system that propagates it out to the systems I access interactively.
• Each upstream community that allows SSH access, usually to gain write access for source control commits. (Again, the public key is often uploaded to a central site and propagated in an automated manner.)

Of course, I need to keep all of these keys secure. I passphrase protect all (ok, most of) the keys, and am careful about access to the private key files. In addition to the keys used from my workstation, I also have separate keys for any shared applications, plus the keys that need to be uploaded to an automation system such as Ansible Tower.

How does my system decide which key to use?

When I generate an SSH key pair, I get prompted for the name of the public key (identity) file with a default of ~/.ssh/id_rsa. I pick a name that hopefully makes as much sense to future me as it does currently. When I use a client command such as ssh or scp, the utility selects a file based on command-line options, a per-host basis in the configuration file, or program defaults:

The ssh man page not only describes the -i option, but also has a section titled AUTHENTICATION which further explains the steps used to determine which key or other method is used.

Command-line options

There are a few options I use on the command line during setup, or for verification and then later in the configuration file for future use. The -i option specifies the key to use and works the same with all of the SSH client utilities, including the ssh, ssh-copy-id, and scp commands:

$ ssh -i ~/.ssh/id_somehubs user@host

This option can be given muliple times to limit which keys to try, if you know it is one of a handful of keys, but I usually only need to specify the exact key.

I also use a handful of other options specified with -o. These options are described in the ssh_config man page. The IdentityFile SSH option can be used instead of -i. The following command has the same result as the one above:

$ ssh -o IdentityFile=~/.ssh/id_somehubs user@host

Other options I use include:

• PreferredAuthentications specifies the order of methods to try. The default generally has five to six options listed with Kerberos first, keys in the middle, and password last. If I know I need to be prompted for a password, such as when copying a new public key to a host, I use -o PreferredAuthentications=password.

• PasswordAuthentication defaults to yes so that if other methods fail, the user will see a password prompt. I sometimes disable this setting to ensure that I am authenticating with a method other than SSH password authentication. If I see a prompt, I know it is a passphrase or Kerberos prompt. I only need to specify PasswordAuthentication=yes if I am trying to override a locally customized configuration file.

• PubkeyAuthentication defaults to yes so that key authentication is attempted. I may set this option to no if I know I need to be prompted for a password, such as to add or replace a key using ssh-copy-id.

• IdentitiesOnly defaults to no, but when set to yes, tells SSH to use only the identity specified on the command line or in the configuration file. The client will not try other identities, even if offered by ssh-agent or a PK11 provider.

Common authentication error

There is a limit on attempts before the SSH server will fail the authentication. When I try to place a key on a new system, I often get a Received disconnect from x.x.x.x port 22:2: Too many authentication failures error message, which means that the client attempted to authenticate with each method and each key and was ultimately disconnected from the server before getting to the final method of prompting for a password.

In the sshd_config file, you can configure MaxAuthTries. It defaults to six. If I have just key and password authentication methods in use, and I have more than five keys, each key is checked in turn until I'm disconnected from the server before I get a chance to enter a password. I don’t always have access to the server-side configuration. Even if I did, I would not change this setting just for the few users that have such a large collection of keys.

When I know I need password authentication, I make use of PubkeyAuthentication=no or PrefferedAuthentication=password to make sure I get prompted for the password. If I have a particular key to use, I can specify the key and set IdentiesOnly=yes so that only that key is tried:

$ ssh-copy-id -i ~/.ssh/id_somehubs -o PreferredAuthentications=password user@host

Configuration file Host entries

To avoid repetitive and lengthy command-line options, I maintain a local configuration file that sets the identity and other options for each destination. As a user, I configure a ~/.ssh/config file. I start by copying the sample from the /etc/ssh directory and then I make use of the ssh_config man page for additional possibilities:

$ cp /etc/ssh/ssh_config ~/.ssh/config
$ man ssh_config
$ vi ~/.ssh/config

For example, I might create a Host section for each destination. Each Host entry supports multiple destinations, as well as wildcards for pattern matching. The ssh_config man page shows many examples, but here’s a particularly useful one for Fedora users:

Host *fedoraproject.org *fedorapeople.org *pagure.io

If you have a different username on different systems, you can add the User option to specify which one. When I connect to one of the hosts listed above, I can just use ssh host instead of ssh user@host, and the correct username will be passed from the configuration file, thanks to:

Host *fedoraproject.org *fedorapeople.org *pagure.io
	User lookitup

You can also add one or more IdentityFile lines for keys used at these sites:

Host *fedoraproject.org *fedorapeople.org *pagure.io
	User lookitup
        IdentityFile ~/.ssh/id_IKnowWhichKey

Then, add any other options for managing the connection. This includes options to enable or disable authentication methods as well as destination ports, environment settings, and proxy commands. You might ultimately end up with:

Host *fedoraproject.org *fedorapeople.org *pagure.io
	User lookitup
        IdentityFile ~/.ssh/id_IKnowWhichKey
	IdentitiesOnly yes

This section of your ~./ssh/config file might end up looking something like this:

A final word on lost keys and key rotation

With multiple keys, I need to determine if all keys were compromised, or if only a single key needs to be rotated. A theft of my laptop would be all keys. If I copy a single key to a new client system and forget to remove it, then I only worry about that one key. Which is exactly why I use different keys for lab testing or any situation where I may need to share a key. My client configuration files then make it easy for me to use a variety of keys on a daily basis.