SSH: Managing SSH Keys with ssh-add and ssh-agent

A complete guide to SSH key management — using ssh-agent and ssh-add to load, list, and remove keys, persisting key loading across sessions, configuring the SSH config file, and troubleshooting common authentication issues.

Introduction

ssh-add is a command-line tool used to add SSH private keys to the SSH authentication agent (ssh-agent). This allows for seamless authentication without re-entering passphrases repeatedly. It is especially useful when managing multiple SSH keys for different servers or services.

The workflow is:

  1. Start ssh-agent once per session
  2. Add your private keys with ssh-add
  3. Connect to any authorized server without entering a passphrase again

Checking If ssh-agent is Running

Before adding keys, verify the SSH agent is running:

bash 
eval "$(ssh-agent -s)"

This starts the agent if it is not already running, and prints its Process ID (PID). The eval command sets the necessary environment variables (SSH_AUTH_SOCK and SSH_AGENT_PID) in the current shell.

md 
Output example:
Agent pid 12345

If the agent is not running, ssh-add commands will fail with "Could not open a connection to your authentication agent."

Adding a Private Key to ssh-agent

bash 
# Add the default private key
ssh-add ~/.ssh/id_rsa

# Add a named key
ssh-add ~/.ssh/company_server

# Add a key from a non-standard location
ssh-add /path/to/custom/key

If the private key is protected with a passphrase, you will be prompted to enter it once. After that, ssh-agent handles authentication transparently.

md 
Output:
Identity added: /home/user/.ssh/id_rsa (user@hostname)

Listing Loaded Keys

Check which keys are currently held in the agent:

bash 
ssh-add -l

This displays the fingerprints and comments of all loaded keys.

md 
Output example:
4096 SHA256:abc123def456... user@hostname (RSA)
2048 SHA256:xyz789ghi012... work-key (RSA)

Removing Keys from ssh-agent

bash 
# Remove a specific key
ssh-add -d ~/.ssh/id_rsa

# Remove ALL keys from the agent
ssh-add -D

Removing keys is useful when you want to revoke temporary access, end a work session, or clean up before handing a machine to someone else.

Using a Key for a Single Session (Without Adding to Agent)

If you want to authenticate with a specific key for one connection without adding it to the agent permanently:

bash 
ssh -i ~/.ssh/id_rsa user@remote_host

This directly specifies the identity file without registering it with ssh-agent. Useful when you do not want the key stored in the agent for the rest of the session.

Storing Keys Permanently Across Reboots

By default, ssh-agent does not persist between system restarts — you need to start it and add keys each time. There are two approaches to automate this.

Option 1: Shell Startup Script

Add the following to ~/.bashrc or ~/.zshrc:

bash 
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_rsa

This starts the agent and loads the key every time a new terminal session opens. Note: this creates a new agent process per terminal, which can accumulate over time.

Option 2: SSH Configuration File

Edit ~/.ssh/config to automatically add keys when connecting to remote hosts:

ini 
Host *
    IdentityFile ~/.ssh/id_rsa
    AddKeysToAgent yes

With AddKeysToAgent yes, whenever a key is used to authenticate, it is automatically added to the running agent. This is the cleaner approach as it only loads keys when they are actually needed.

SSH Config File: Per-Host Configuration

The ~/.ssh/config file can define per-host settings to simplify connections:

ini 
# Default settings for all hosts
Host *
    AddKeysToAgent yes
    IdentityFile ~/.ssh/id_rsa

# Specific host configuration
Host webserver
    HostName 192.168.1.100
    User deploy
    Port 2222
    IdentityFile ~/.ssh/deploy_key

Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_key

With this configuration, instead of typing:

bash 
ssh -p 2222 -i ~/.ssh/deploy_key deploy@192.168.1.100

You can simply type:

bash 
ssh webserver

Managing Multiple Keys for Different Services

A common pattern in professional environments is maintaining separate keys for different purposes:

bash 
# Generate keys for different purposes
ssh-keygen -t rsa -b 4096 -f ~/.ssh/github_key -C "github"
ssh-keygen -t rsa -b 4096 -f ~/.ssh/work_server -C "work-production"
ssh-keygen -t rsa -b 4096 -f ~/.ssh/personal_vps -C "personal-vps"

Then configure ~/.ssh/config to use the appropriate key per host:

ini 
Host github.com
    IdentityFile ~/.ssh/github_key

Host prod-server.company.com
    User deploy
    IdentityFile ~/.ssh/work_server

Host my-vps.example.com
    User root
    IdentityFile ~/.ssh/personal_vps

Troubleshooting ssh-add

SSH Agent Not Running

bash 
# Error: Could not open a connection to your authentication agent
# Fix: Start the agent
eval "$(ssh-agent -s)"

Incorrect Key Permissions

SSH is strict about key file permissions. Private keys must be readable only by the owner:

bash 
# Fix permissions on private key
chmod 600 ~/.ssh/id_rsa

# Fix permissions on .ssh directory
chmod 700 ~/.ssh

If permissions are too open (e.g., world-readable), SSH will refuse to use the key and display a warning.

Authentication Still Failing

bash 
# Debug the SSH connection with verbose output
ssh -vvv user@remote_host

# Look for lines like:
# "Offering public key: /home/user/.ssh/id_rsa"
# "Server accepted the key" or "Permission denied"

The debug output shows exactly which keys were offered and whether the server accepted them.

Agent Not Available in New Sessions

If ssh-add -l returns "The agent has no identities" after restarting a terminal, the agent environment variables were not inherited. Use the startup script approach or the SSH config AddKeysToAgent option to ensure keys are available.

Key Security Best Practices

md 
Private Key Security:
├── Always protect private keys with a strong passphrase
├── Use chmod 600 on private key files
├── Never copy private keys to remote servers
├── Use separate keys for different access levels
└── Rotate keys periodically

Agent Security:
├── Lock the agent when leaving your desk: ssh-add -x
├── Use ssh-add -t <seconds> to add keys with a timeout
│   ssh-add -t 3600 ~/.ssh/id_rsa  (expires in 1 hour)
└── Remove all keys when done: ssh-add -D