SSH (Secure Shell) is the standard way to open an encrypted terminal session on a remote Linux server. In this tutorial you will connect to a server for the first time with a password, verify its host key, switch to key-based authentication with an Ed25519 key pair, and save the connection in an SSH config file so you can log in with a short alias. The client steps work on Linux, macOS and Windows 10/11; the server can be any Linux distribution running OpenSSH, such as Ubuntu 24.04.

Prerequisites

To follow this guide you need:

  • A Linux server with the OpenSSH server running, for example a CubePath VPS with Ubuntu 24.04.
  • The server's public IP address (shown as your_server_ip in this guide), the login user (often root on a new server) and its password or an existing key.
  • A local computer running Linux, macOS, or Windows 10/11.
  • TCP port 22 reachable from your network (or the custom port your server uses).

Step 1 - Checking the SSH client on your computer

All modern desktop systems ship an OpenSSH client, so there is usually nothing to install. Open a terminal (on Windows, open PowerShell or Windows Terminal) and check the version:

bash
ssh -V
output
OpenSSH_9.6p1 Ubuntu-3ubuntu13.5, OpenSSL 3.0.13 30 Jan 2024

Any OpenSSH 8.x or newer is fine. If the command is not found:

  • Debian/Ubuntu desktop: sudo apt install openssh-client
  • Fedora/Rocky/RHEL: sudo dnf install openssh-clients
  • Windows: open Settings > System > Optional features, choose Add a feature and install OpenSSH Client, then open a new PowerShell window.

The rest of this tutorial uses the same ssh, ssh-keygen and scp commands on every platform. The only difference on Windows is how the public key is copied in Step 4.

Step 2 - Connecting for the first time with a password

The basic syntax is ssh user@host. Replace your_server_ip with the address of your server:

bash
ssh root@your_server_ip

If the server listens on a non-standard port, add -p:

bash
ssh -p 2222 root@your_server_ip

On the first connection the client does not know the server yet and shows its host key fingerprint:

output
The authenticity of host '203.0.113.10 (203.0.113.10)' can't be established.
ED25519 key fingerprint is SHA256:Qm3uCz2XlY0wq9mH6bJ0V7yE0mZt3Q9f1s0bT2yN8aE.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

This prompt protects you against someone intercepting the connection. To be certain you are talking to your server, compare the fingerprint with the one the server itself reports. Open the web console of your server (on CubePath, the VNC console of the VPS in the dashboard), log in, and run:

bash
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
output
256 SHA256:Qm3uCz2XlY0wq9mH6bJ0V7yE0mZt3Q9f1s0bT2yN8aE root@server (ED25519)

If both fingerprints match, type yes and press ENTER. The client stores the key in ~/.ssh/known_hosts and will warn you loudly if it ever changes. Then type the password; nothing is echoed while you type.

A successful login ends at a shell prompt on the server:

output
Welcome to Ubuntu 24.04.3 LTS (GNU/Linux 6.8.0-79-generic x86_64)
...
root@server:~#

Type exit or press CTRL+D to close the session.

Step 3 - Generating an SSH key pair

Passwords can be guessed and are sent to the server on every login. A key pair is safer: the private key never leaves your computer, and the server only stores the public half. Ed25519 is the recommended key type today; it is short, fast and supported by every current OpenSSH server.

Run this on your local computer, not on the server:

bash
ssh-keygen -t ed25519 -C "[email protected]"

Press ENTER to accept the default location and then type a passphrase. The passphrase encrypts the private key on disk, so a stolen laptop does not mean a stolen server.

output
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/sammy/.ssh/id_ed25519):
Enter passphrase for "/home/sammy/.ssh/id_ed25519" (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/sammy/.ssh/id_ed25519
Your public key has been saved in /home/sammy/.ssh/id_ed25519.pub

This creates two files:

FileContentsShare it?
~/.ssh/id_ed25519Private keyNever
~/.ssh/id_ed25519.pubPublic keyYes, this goes on the server

On Windows the same files are created in C:\Users\your_user\.ssh\.

Step 4 - Copying the public key to the server

The server accepts a key when its public half is listed in ~/.ssh/authorized_keys of the user you log in as.

On Linux and macOS

ssh-copy-id connects with your password one last time and appends the key with the correct permissions:

bash
ssh-copy-id root@your_server_ip

For a custom port use ssh-copy-id -p 2222 root@your_server_ip.

output
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
[email protected]'s password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

On Windows (PowerShell)

Windows does not include ssh-copy-id. This one-liner reads your public key and appends it on the server, creating the directory with the right permissions if needed:

powershell
type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh root@your_server_ip "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Testing the key

Connect again:

bash
ssh root@your_server_ip

This time you are asked for the key's passphrase (a local prompt) instead of the server password:

output
Enter passphrase for key '/home/sammy/.ssh/id_ed25519':

If you still get a password: prompt, the key was not accepted. See the Troubleshooting section before continuing.

Step 5 - Caching the passphrase with ssh-agent

Typing the passphrase for every connection gets tedious. ssh-agent keeps the decrypted key in memory for your session.

On Linux, most desktop sessions already run an agent. Add the key with:

bash
ssh-add ~/.ssh/id_ed25519

On macOS, store the passphrase in the Keychain so it survives reboots:

bash
ssh-add --apple-use-keychain ~/.ssh/id_ed25519

On Windows, the agent is a service that is disabled by default. In an administrator PowerShell:

powershell
Get-Service ssh-agent | Set-Service -StartupType Automatic
Start-Service ssh-agent

Then, in a normal PowerShell window:

powershell
ssh-add $env:USERPROFILE\.ssh\id_ed25519

Check which keys the agent holds:

bash
ssh-add -l
output
256 SHA256:3Jd0N1m8gq7sK7c0Qv9eZ2rVb4P8XyWmT5uLkHcA1sE [email protected] (ED25519)

Step 6 - Saving connections in ~/.ssh/config

Instead of remembering IP addresses, users and ports, describe each server once in the client config file. Create or open it on your local computer:

bash
nano ~/.ssh/config

Add a block per server:

text
Host web1
    HostName your_server_ip
    User root
    Port 22
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes
    ServerAliveInterval 60

What each line does:

  • Host web1: the alias you will type.
  • HostName, User, Port: the real connection details.
  • IdentityFile and IdentitiesOnly yes: offer only this key, which avoids "Too many authentication failures" when the agent holds many keys.
  • ServerAliveInterval 60: sends a keepalive every 60 seconds so idle sessions are not dropped by NAT routers.

On Linux and macOS, make sure only you can read the file:

bash
chmod 600 ~/.ssh/config

Now connect with the alias:

bash
ssh web1

The alias works with the other OpenSSH tools too, for example to copy a file to the server's /tmp directory:

bash
scp ./backup.tar.gz web1:/tmp/

Step 7 - Disabling password logins (optional)

Once key login works, you can stop the server from accepting passwords at all, which ends password brute-force attempts. Do this only after confirming in Step 4 that your key works, and keep your current session open while you test.

On the server, create a drop-in file. OpenSSH uses the first value it reads for each option, and files in /etc/ssh/sshd_config.d/ are read in alphabetical order before the main file, so a name starting with 00- takes precedence over files such as 50-cloud-init.conf:

bash
sudo nano /etc/ssh/sshd_config.d/00-keys-only.conf
text
PasswordAuthentication no
KbdInteractiveAuthentication no

Check the syntax and the effective values, then restart the service (ssh on Ubuntu and Debian, sshd on Rocky Linux and other RHEL-based systems):

bash
sudo sshd -t
sudo sshd -T | grep -Ei '^(passwordauthentication|kbdinteractiveauthentication)'
sudo systemctl restart ssh
output
passwordauthentication no
kbdinteractiveauthentication no

From a new local terminal, confirm that the server now refuses passwords:

bash
ssh -o PubkeyAuthentication=no root@your_server_ip
output
[email protected]: Permission denied (publickey).

A normal ssh root@your_server_ip with your key should still work.

Troubleshooting

Connection timed out: the packets never reach SSH. Check the IP address, that the server is running, and that a firewall (UFW, firewalld, or a network firewall in front of the server) allows the SSH port.

Connection refused: the server is reachable but nothing listens on that port. You may be using the wrong port, or the SSH service is stopped. From the web console, run sudo systemctl status ssh (or sshd on RHEL-based systems) and sudo ss -tlnp | grep ssh.

Permission denied (publickey): the server did not accept your key. Run the client in verbose mode to see which keys it offers:

bash
ssh -v root@your_server_ip

On the server, the usual causes are wrong ownership or permissions. Fix them for the target user:

bash
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

The server-side reason is logged in the journal: sudo journalctl -u ssh -n 50 (-u sshd on RHEL-based systems).

WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!: the server presents a different host key than the one you saved. This is expected after reinstalling the server with the same IP. If you know that is the case, remove the old entry and connect again, verifying the new fingerprint as in Step 2:

bash
ssh-keygen -R your_server_ip

If you did not reinstall anything, do not connect: investigate first.

Conclusion

You can now connect to your server over SSH, verify its identity, log in with an Ed25519 key protected by a passphrase, and reach it with a short alias. Good next steps are creating a non-root sudo user and hardening the server (firewall, Fail2Ban, automatic updates), changing the default SSH port if you want less log noise, and adding a separate key for each computer you work from so you can revoke one without affecting the others.