Setting up SSH Key management for multiple servers with Conjur

The SSH or Secure Shell protocol has become the industry standard for logging into one computer from another. Engineers use SSH to manage their network infrastructure and perform maintenance and troubleshooting on remote servers. As more organizations adopt the DevOps model, the use of SSH to support operations has created additional security challenges for organizations.

In this article, we’re going to talk about some of the challenges associated with using SSH within an organization. We’ll then discuss strategies, including using tools like Conjur to streamline SSH key management across multiple servers, which empower and enable engineers to do their jobs while also protecting your organization against security threats.

Defining an SSH Management Plan

SSH best practices recommend the use of SSH keys for authentication instead of usernames and passwords. SSH keys are long strings that are extremely difficult to crack, but are nearly impossible for engineers to remember.

Each engineer should have a unique SSH key for a server or group of servers they support. Unique keys support access management and ensure that the organization can monitor and audit access to servers as needed. Different groups of servers require different SSH keys as well, which means that as infrastructure grows and engineering teams grow, the number of SSH keys proliferates exponentially.

SSH best practices also recommend regular rotation of keys, which means that you should also be periodically cycling your ever-growing collection of keys.

An effective SSH management plan should support:

  • Assignment of SSH keys at an individual level
  • Secure storage of keys
  • Scheduled rotation of keys
  • Active management of all keys within your organization

SSH Key Management With Conjur

Conjur from CyberArk is the ideal SSH key management system. The system provides a secure and robust secret store for your organization and allows you to manage keys through policy. Comprehensive auditing capabilities allow you to see who has accessed keys both successfully and unsuccessfully.

Storing your organization’s SSH keys in a secret store eliminates the need for engineers to maintain a set of keys on their workstation, and facilitates periodic rotation of keys with minimal effect on productivity.

Let’s walk through the necessary steps of setting up an SSH key management system with Conjur, and we’ll wrap it up with a proof of concept that you can use to implement a workable SSH key management system within your organization.

The Conjur site includes an excellent series of tutorialsdesigned to get a Conjur instance up and running in short order. If you walk through the first unit, you’ll be ready to try out this system as we go through.

Policies and Administration

For this example, let’s consider that I have an SSH key to access a collection of database servers. We’ll walk through this with one example, but the beauty of the system is that it’s scalable. Once we have our Conjur instance up, the next step is to set up a policy that defines me as a user and creates a variable to store the SSH key for the database servers.

I’ll be creating this policy manually, but this process could be easily automated and connected with resource management systems within the enterprise.

- !policy
  id: MMackrorySSHKeys

  # Define an admin, the user and ssh keys
  - !user SSHKeyAdmin
  - !user mmackr
  - !variable database

  # Give read and write permissions to the administrator.
  - !permit
    role: !user SSHKeyAdmin
    privileges: [read, update, execute]
    resource: !variable database

  # Give read permissions to the service account to fetch the API Key.
  - !permit
    role: !user mmackr
    privileges: [read, execute]
    resource: !variable database


Now that we’ve defined the policy, we’ll log into Conjur as the administrator and execute it. We login to the admin account.


$ docker-compose exec client conjur authn login -u admin

Please enter admin’s password (it will not be echoed):

Logged in

Next, we’ll load the policy we defined above.

$ docker-compose exec client conjur policy load root policy/mmackr_database_key.yml

Loaded policy 'root'


  "created_roles": {

    "ConjurDemo:user:SSHKeyAdmin@MMackrorySSHKeys": {

      "id": "ConjurDemo:user:SSHKeyAdmin@MMackrorySSHKeys",

      "api_key": "34k7f2c37mp3rmw5mxcf2qwfyg62p8jxak3nq1ce73tgtycp146t4xj"


    "ConjurDemo:user:mmackr@MMackrorySSHKeys": {

      "id": "ConjurDemo:user:mmackr@MMackrorySSHKeys",

      "api_key": "39kamky14rdax73eessnt28d5q1z1ap59qt1maacf818mk2r23dybv7k"



  "version": 1


We now have an administrative user with read and write capabilities and a user with read capabilities. We can also define the administrative user as a host, and an automated system could assume the role.

Let’s log out as the administrative user for the account, and log in as the administrative user for the policy.

$ docker-compose exec client conjur authn logout

Logged out

$ docker-compose exec client conjur authn login -u SSHKeyAdmin@MMackrorySSHKeys

Please enter SSHKeyAdmin@MMackrorySSHKeys's password (it will not be echoed):

Logged in

As the administrative user, we can load the SSH key into the variable. The SSH key is in a file called mmackr_database.pem. Conjur responds with ‘Value added’ after the successful import of the file.

$ docker-compose exec client conjur variable values add MMackrorySSHKeys/database "$(cat mmackr_database.pem)"  

Value added

Let’s see if we can retrieve the value from our local workstation using the read-only account we set up.

Interacting with Conjur

The first step in interacting with Conjur is to authenticate ourselves and retrieve an access token. We’ll use the key we created earlier to do this. From our local workstation, we’ll issue a curl command to authenticate ourselves and put the result into a file. I’m running my Conjur instance with AWS, so I’ve put the public URL in as the host.

Conjur API Documentation

$ curl -d "39kamky14rdax73eessnt28d5q1z1ap59qt1maacf818mk2r23dybv7k" -k > access_token

If the authentication attempt succeeds, I should be able to see the token in theaccess_token file.

$ cat access_token



Next, we’ll use that token to retrieve the PEM  (Privacy Enhanced Mail) file from the secret store. The first thing we need to do is encode the token with base64. We’ll save the result in an environment variable.

$ SESSION_TOKEN=$(cat ./access_token | base64 | tr -d '\r\n')

With a valid and encoded token, we can now execute a curl command to retrieve the SSH key from the secure data store.

$ curl -s -k -H "Content-Type: application/json" -H "Authorization: Token token=\"$SESSION_TOKEN\""

Finally, if everything worked, you should see the contents of the SSH key displayed on the screen in a similar format to the one below, but likely much longer.






CLI Proof of Concept

Let’s see if we can build on the ideas above and create a process to simplify “SSH’ing” into a server. Since each user has unique information that remains constant, we’ll start with a configuration file. I’ll save mine as connect.config




Using the values from the config, we can create a shell script, which asks the user for a host and a key. The script authenticates with Conjur, retrieves the SSH key, creates the connection and then removes the SSH key. I’ll save this as


main() {

read -p 'Host: ' HOST

read -p 'Key: ' KEY


. ./connect.config


curl -d "$TOKEN" -k$USER@$REPO/authenticate > access_token

SESSION_TOKEN=$(cat ./access_token | base64 | tr -d '\r\n')

curl -s -k -H "Content-Type: application/json" -H "Authorization: Token token=\"$SESSION_TOKEN\""$KEY > tempcreds.pem

ssh -i tempcreds.pem ec2-user@$HOST

rm tempcreds.pem


main "$@"


Now, if I want to connect to a server using my secured ‘database’ key, I can just run the script.

$ sh


Key: database

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

Dload  Upload   Total   Spent    Left  Speed

100   656    0   600  100    56    600     56  0:00:01 --:--:--  0:00:01  2791

Last login: Mon Jul  8 15:45:30 2019 from


__|  __|_  )

_|  (     /   Amazon Linux AMI


[ec2-user@ip-172-31-30-166 ~]$


This solution isn’t ready to be rolled out to your user base, but it illustrates how you can leverage Conjur to secure your SSH keys better and simplify their management. With this solution, I can rotate the SSH keys, and the user has instant access to the new keys. If we need to revoke access for a user, we rotate their keys and remove access from within the Conjur policy.

Developing the Idea Further
If you’re looking for additional resources and help, I’d highly recommend the CyberArk Commons community. You can network with other engineers, ask for help, and contribute to the future of Conjur Open Source.

For more information on Conjur itself, I recommend the Getting Started and Documentation sections of the website.