Creating SSH Keys In A Different Way

2026-06-23 Linux

I have some different habits when it comes to technology, which has led me to develop some “weird” workflows. To be honest, anyone who is in this field for some time has such habits. Today I’ll talk about generating SSH keys.

The usual suspects

Consider this scenario

You have a VPS to which you connect via SSH. It’s a good habit to turn off password authentication to improve the security of your server. So, being a responsible sys-admin, you generate an SSH key-pair from your PC and transfer it to your VPS using ssh-copy-id. Once you see your password-less login working, you turn off password-based authentication (PBA). Your login is now seamless.

Good bye pesky hackers! You will not be missed. No more login issues.

Or is it really?

Where is gets messy

After a few weeks, you decide that another PC in your fleet needs to access the same VPS. Now you generate a new SSH key-pair at the other PC, but you can’t transfer it the VPS using ssh-copy-id since you have turned off PBA at the VPS. Only the original machine can access it (since it has the private key). Now, you can enable PBA again, but it’s been a while and you forgot what the original settings were. When you try to enable PBA, you mess up something and suddenly, you have lost SSH access to your server. And everytime you want to add another machine, the same problem crops up again.

So, what do you do?

(Note: Regaining access to a VPS which has no SSH access is a separate topic)

Motivation behind the solution

I don’t know what the right solution is, but I love the fact that Linux gives us the freedom to chose our solution. It can be a bad solution, or it can be a really bad solution, but it’s your own way of solving a problem.

Before I come to the solution, let me state what I want in the solution

Number of actions to enable passwordless login at the other machine should be minimal (Least cognitive load on the user)
Should not involve changing any configuration file if possible

I discovered my preferred way of solving this by adopting what cloud providers do. When cloud providers (AWS, Oracle, etc) allocate a VM to you, they provide you with a key file. You don’t generate a key, or need to transfer any key, or even modify any files. The cloud providers simply give you a private key. You download the private key on any number of machines and use it to login. Let us try to replicate it for our server.

Step 1: Generate a fresh SSH key-pair on your local machine

ssh-keygen -t ed25519 -C <SERVER>

This will generate two files

<SERVER>.key - The private key
<SERVER>.pub - The public key

Step 2: Transfer the public key to the server

This should be obvious but you need to have access to your server before you generate the key pair.

Copy the contents of <SERVER>.pub and place it inside ~/.ssh/authorized_keys on the server. If the file already contains other public keys, place the copied contents in the last line of the file.

Step 3: Transfer the private key to your preferred machine

You can use any mechanism to transfer the private key (ssh, sftp etc).

chmod 600 <SERVER>.key

Once you have set permissions on the private key, you can login with your key

ssh <USER>@<SERVER_IP> -i <SERVER>.key

You should be able to figure this out. But before you turn it off, keep one SSH session active. In case you used some incorrect settings, you can revert it from this backup SSH session.

Step 5: The end

If the key-based login is working, congratulations. You can restart the sshd service on the VPS.

sudo systemctl restart sshd.service

Keep the private key secure. You can now transfer the key file to any other machine and login without affecting SSH settings on the VPS.

Conclusion

I like this solution because it’s the most hassle-free solution. I just need to transfer a single key file to my machine of choice, and there is no specific command to run or setting to change. There can be other solutions as well, such as involving ssh-copy-id, but this suites me. Also, relying too much on convenient commands often makes you forget the underlying mechanism of certain actions (This is purely my personal opinion).

A downside of using a single keyfile is that if any one of your machine is compromised, you need to discard that key from all machines in your fleet and create a new key. It’s also hard to audit the logins since all logins to the VPS will have the same signature, since all machines are using the same key-pair. But it’s not a huge issue in my opinion.