How to Connect to Your Server via SSH

Secure Shell (SSH) is the fundamental protocol for secure remote server administration, allowing system administrators and developers to connect to Linux servers securely over the internet. This comprehensive guide will walk you through everything you need to know about SSH connections, from basic setup to advanced authentication methods, ensuring you can establish secure and reliable connections to your servers.

Table of Contents

Prerequisites

Before connecting to your server via SSH, ensure you have:

  • A Linux server (VPS, dedicated server, or cloud instance) with SSH server installed and running
  • Server IP address or hostname
  • Valid user credentials (username and password or SSH key)
  • SSH client installed on your local machine (Linux, macOS, or Windows)
  • Network connectivity between your local machine and the server
  • Server SSH port open (default: 22)
  • Basic command-line knowledge

Understanding SSH Protocol

SSH (Secure Shell) is a cryptographic network protocol that provides secure communication over unsecured networks. Unlike older protocols like Telnet or FTP, SSH encrypts all data transmitted between your computer and the server, including passwords, commands, and outputs.

Key features of SSH:

  • Encryption: All communication is encrypted using strong cryptographic algorithms
  • Authentication: Supports multiple authentication methods (password, public key, certificate)
  • Integrity: Ensures data hasn't been tampered with during transmission
  • Port forwarding: Allows secure tunneling of other protocols through SSH
  • File transfer: Enables secure file operations via SCP and SFTP

SSH operates on a client-server architecture. The SSH server (typically sshd) runs on your remote server, listening for incoming connections on port 22 by default. The SSH client initiates the connection from your local machine.

Step 1: Installing SSH Client on Your Local Machine

The installation process varies depending on your operating system.

Linux (Ubuntu/Debian)

Most Linux distributions come with an SSH client pre-installed. To verify or install:

# Check if SSH client is installed
ssh -V

If not installed, install the OpenSSH client:

# Update package repository
sudo apt update

# Install OpenSSH client
sudo apt install openssh-client -y

Linux (CentOS/Rocky Linux/RHEL)

# Check if SSH client is installed
ssh -V

# Install if needed
sudo dnf install openssh-clients -y
# Or on older versions:
sudo yum install openssh-clients -y

macOS

macOS includes OpenSSH client by default. You can verify by opening Terminal and running:

# Check SSH client version
ssh -V

Windows

Windows 10 (version 1809+) and Windows 11 include OpenSSH client by default. To enable it:

  1. Open Settings > Apps > Optional Features
  2. Search for "OpenSSH Client"
  3. Click "Install" if not already installed

Alternatively, you can use third-party SSH clients like PuTTY, MobaXterm, or Windows Subsystem for Linux (WSL).

For PuTTY installation:

  1. Download PuTTY from the official website (https://www.putty.org/)
  2. Run the installer
  3. Launch PuTTY from the Start menu

Step 2: Obtaining Your Server Connection Details

Before connecting, gather the following information from your hosting provider or server documentation:

  • Server IP Address: IPv4 (e.g., 192.168.1.100) or IPv6 address
  • Username: The account you'll use to log in (commonly root, ubuntu, admin, or custom username)
  • Port: SSH port number (default is 22, but may be customized for security)
  • Authentication method: Password or SSH key path

Most VPS providers send this information via email when you provision a new server. If you're using cloud platforms like AWS, DigitalOcean, or Linode, you can find these details in your control panel.

Step 3: Connecting via SSH with Password Authentication

The most straightforward method to connect to your server is using password authentication.

Basic SSH Connection Syntax

# Standard SSH connection command
ssh username@server_ip_address

# Example with specific IP
ssh [email protected]

# Example with hostname
ssh [email protected]

# Specify custom port
ssh -p 2222 username@server_ip_address

When you run this command:

  1. The SSH client initiates a connection to the server
  2. On first connection, you'll see a fingerprint verification message
  3. Type yes to accept and add the server to known hosts
  4. Enter your password when prompted
  5. Upon successful authentication, you'll be logged into the remote server

First Connection Example

# First-time connection
ssh [email protected]

You'll see a message like:

The authenticity of host '203.0.113.10 (203.0.113.10)' can't be established.
ECDSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Type yes and press Enter. This adds the server's fingerprint to ~/.ssh/known_hosts, preventing man-in-the-middle attacks on subsequent connections.

Why fingerprint verification matters: This security measure ensures you're connecting to the genuine server and not an imposter. Always verify the fingerprint matches what your hosting provider shows in their control panel.

Connecting with Different Port

If your SSH server runs on a non-standard port:

# Connect to SSH server on port 2222
ssh -p 2222 username@server_ip_address

# Example
ssh -p 2222 [email protected]

The -p flag specifies the port number. This is common in security-hardened environments where administrators change the default port to reduce automated attacks.

Step 4: Setting Up SSH Key Authentication

SSH key authentication is more secure than password-based authentication and is the recommended method for server access. It uses public-key cryptography to authenticate without transmitting passwords.

Generating SSH Key Pair

# Generate RSA key pair (4096-bit for maximum security)
ssh-keygen -t rsa -b 4096 -C "[email protected]"

# Or generate Ed25519 key (modern, more secure, recommended)
ssh-keygen -t ed25519 -C "[email protected]"

When prompted:

  1. File location: Press Enter to accept default (~/.ssh/id_rsa or ~/.ssh/id_ed25519)
  2. Passphrase: Enter a strong passphrase for additional security (optional but recommended)

This creates two files:

  • Private key: ~/.ssh/id_rsa or ~/.ssh/id_ed25519 (never share this)
  • Public key: ~/.ssh/id_rsa.pub or ~/.ssh/id_ed25519.pub (safe to share)

Why use SSH keys: Key-based authentication is immune to brute-force attacks, doesn't require typing passwords, and can be easily revoked without changing credentials.

Copying Public Key to Server

Method 1: Using ssh-copy-id (recommended)

# Copy public key to server
ssh-copy-id username@server_ip_address

# With custom port
ssh-copy-id -p 2222 username@server_ip_address

# With specific key file
ssh-copy-id -i ~/.ssh/id_ed25519.pub username@server_ip_address

This command automatically appends your public key to ~/.ssh/authorized_keys on the server with correct permissions.

Method 2: Manual copying

If ssh-copy-id is unavailable:

# Display your public key
cat ~/.ssh/id_rsa.pub

# Copy the output, then SSH to server and run:
mkdir -p ~/.ssh
echo "your_public_key_content_here" >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Method 3: One-liner approach

# Copy key using cat and SSH
cat ~/.ssh/id_rsa.pub | ssh username@server_ip_address "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys"

Connecting with SSH Key

After copying your public key:

# Connect using default private key
ssh username@server_ip_address

# Specify custom private key
ssh -i ~/.ssh/custom_key username@server_ip_address

# Example with all options
ssh -i ~/.ssh/id_ed25519 -p 2222 [email protected]

If you set a passphrase on your private key, you'll be prompted to enter it. This provides an additional layer of security.

Step 5: Configuring SSH Client for Easier Connections

Creating an SSH configuration file simplifies connection commands and stores preferences.

Creating SSH Config File

# Create or edit SSH config file
nano ~/.ssh/config

Add server configurations:

# Production server configuration
Host production
    HostName 203.0.113.10
    User root
    Port 22
    IdentityFile ~/.ssh/id_ed25519
    ServerAliveInterval 60
    ServerAliveCountMax 3

# Development server with custom port
Host dev-server
    HostName dev.example.com
    User admin
    Port 2222
    IdentityFile ~/.ssh/dev_key
    ForwardAgent yes

# Staging environment
Host staging
    HostName 203.0.113.20
    User deploy
    IdentityFile ~/.ssh/staging_key
    StrictHostKeyChecking ask

Set proper permissions:

# Secure the config file
chmod 600 ~/.ssh/config

Now you can connect using the host alias:

# Connect to production server
ssh production

# Connect to development server
ssh dev-server

# Connect to staging
ssh staging

Configuration options explained:

  • Host: Alias name for the connection
  • HostName: Actual server IP or domain
  • User: Default username
  • Port: SSH port number
  • IdentityFile: Path to private key
  • ServerAliveInterval: Sends keepalive packets every N seconds
  • ServerAliveCountMax: Number of keepalive packets before disconnection
  • ForwardAgent: Allows using local SSH keys on remote server
  • StrictHostKeyChecking: Controls host key verification (ask, yes, no)

Step 6: Advanced SSH Connection Options

SSH Agent for Key Management

SSH agent stores private keys in memory, eliminating repeated passphrase prompts:

# Start SSH agent
eval "$(ssh-agent -s)"

# Add private key to agent
ssh-add ~/.ssh/id_ed25519

# List loaded keys
ssh-add -l

# Remove all keys from agent
ssh-add -D

Connection Multiplexing

Reuse existing SSH connections for faster subsequent connections:

# Add to ~/.ssh/config
Host *
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h-%p
    ControlPersist 600

Create socket directory:

# Create directory for control sockets
mkdir -p ~/.ssh/sockets
chmod 700 ~/.ssh/sockets

SSH with Command Execution

Execute commands without interactive login:

# Run single command
ssh username@server "uptime"

# Run multiple commands
ssh username@server "df -h && free -m"

# Execute script on remote server
ssh username@server 'bash -s' < local_script.sh

# Run command with sudo
ssh username@server "sudo systemctl restart nginx"

SSH Tunneling (Port Forwarding)

Local port forwarding (access remote service locally):

# Forward remote MySQL to local port 3307
ssh -L 3307:localhost:3306 username@server

# Access remote web service locally
ssh -L 8080:localhost:80 username@server

Remote port forwarding (expose local service to remote):

# Make local web server accessible from remote server
ssh -R 8080:localhost:80 username@server

Dynamic port forwarding (SOCKS proxy):

# Create SOCKS proxy on port 1080
ssh -D 1080 username@server

Verbose Mode for Debugging

Enable verbose output to troubleshoot connection issues:

# Basic verbose mode
ssh -v username@server

# More verbose (level 2)
ssh -vv username@server

# Maximum verbosity (level 3)
ssh -vvv username@server

Verification

After establishing an SSH connection, verify you're successfully connected:

Check Current User and Hostname

# Display current user
whoami

# Display hostname
hostname

# Display full system information
uname -a

# Show who is logged in
who

Verify Connection Details

# Check SSH environment variables
echo $SSH_CONNECTION
echo $SSH_CLIENT

# View last login information
last -a username

# Check active SSH sessions
w

Test Permissions

# Verify you can execute commands
ls -la /root

# Test sudo access (if applicable)
sudo -v

Troubleshooting

Connection Refused

Problem: ssh: connect to host X.X.X.X port 22: Connection refused

Solutions:

# Verify SSH service is running on server (from server console/panel)
sudo systemctl status sshd

# Start SSH service if stopped
sudo systemctl start sshd

# Enable SSH to start on boot
sudo systemctl enable sshd

# Check if SSH is listening on correct port
sudo ss -tlnp | grep sshd

# Verify firewall allows SSH
sudo ufw status
sudo firewall-cmd --list-all

Connection Timeout

Problem: ssh: connect to host X.X.X.X port 22: Connection timed out

Solutions:

  • Verify the server IP address is correct
  • Check if your firewall blocks outbound port 22
  • Confirm the server firewall allows inbound SSH connections
  • Verify the server is online (ping test)
  • Check if your ISP blocks port 22
# Test connectivity to server
ping server_ip_address

# Test if port 22 is open
nc -zv server_ip_address 22
telnet server_ip_address 22

Permission Denied (Public Key)

Problem: Permission denied (publickey)

Solutions:

# Verify public key is in authorized_keys on server
cat ~/.ssh/authorized_keys

# Check permissions on server
ls -la ~/.ssh
# Should be: drwx------ (700) for .ssh directory
# Should be: -rw------- (600) for authorized_keys file

# Fix permissions on server
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

# Verify you're using correct private key
ssh -i ~/.ssh/id_ed25519 -v username@server

# Check SSH agent has your key
ssh-add -l

Host Key Verification Failed

Problem: WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!

Solutions:

# Remove old host key (only if you know the server was reinstalled)
ssh-keygen -R server_ip_address

# Or manually edit known_hosts
nano ~/.ssh/known_hosts
# Remove the line for the affected server

Security warning: Only remove host keys if you're certain the server was legitimately changed (reinstalled, IP reassigned). This warning could indicate a man-in-the-middle attack.

Slow SSH Connection

Problem: SSH connection takes 10-30 seconds to establish

Solutions:

# Disable DNS lookup on server (add to /etc/ssh/sshd_config)
UseDNS no

# Disable GSSAPI authentication in client
ssh -o GSSAPIAuthentication=no username@server

# Add to ~/.ssh/config for permanent fix
Host *
    GSSAPIAuthentication no

# Restart SSH service after config changes
sudo systemctl restart sshd

Best Practices

Security Recommendations

  1. Always use SSH key authentication instead of passwords
  2. Protect private keys with strong passphrases
  3. Use separate keys for different servers or purposes
  4. Regularly rotate SSH keys (annually or when employees leave)
  5. Disable root login and use sudo for administrative tasks
  6. Change default SSH port to reduce automated attacks
  7. Implement fail2ban or similar tools to block brute-force attempts
  8. Use firewall rules to restrict SSH access to specific IP addresses
  9. Enable two-factor authentication (2FA) for critical servers
  10. Keep SSH client and server updated to patch vulnerabilities

Configuration Security

Add these settings to server's /etc/ssh/sshd_config:

# Disable password authentication (use keys only)
PasswordAuthentication no

# Disable root login
PermitRootLogin no

# Allow only specific users
AllowUsers admin deploy

# Use protocol 2 only
Protocol 2

# Reduce login grace time
LoginGraceTime 30

# Maximum authentication attempts
MaxAuthTries 3

# Disable empty passwords
PermitEmptyPasswords no

Performance Tips

  1. Use connection multiplexing to speed up multiple connections
  2. Enable compression for slow connections: ssh -C username@server
  3. Use SSH agent to avoid repeated passphrase entry
  4. Configure keep-alive to prevent disconnections: ServerAliveInterval 60
  5. Limit cipher choices to faster algorithms for trusted networks

Maintenance Advice

  1. Monitor SSH logs regularly: sudo tail -f /var/log/auth.log
  2. Audit authorized_keys files periodically for unauthorized keys
  3. Document SSH configurations for team members
  4. Backup private keys securely (encrypted storage)
  5. Test SSH access after any server or firewall changes
  6. Use version control for SSH config files
  7. Set up alerts for failed login attempts

Connection Management

# Set connection timeout in ~/.ssh/config
Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    ConnectTimeout 10

# Use connection multiplexing
Host *
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h-%p
    ControlPersist 10m

Conclusion

SSH is the backbone of secure remote server administration. By following this comprehensive guide, you've learned how to connect to your server using both password and key-based authentication, configure SSH client settings for convenience, and implement security best practices.

Key takeaways:

  • SSH provides encrypted, secure communication for remote server access
  • Key-based authentication is significantly more secure than passwords
  • SSH config files simplify connection management for multiple servers
  • Regular security audits and updates are essential for maintaining secure access
  • Connection multiplexing and SSH agent improve workflow efficiency

Now that you've mastered SSH connections, you can proceed to initial server security configuration, user management, and other administrative tasks. Remember to always prioritize security and follow best practices when managing server access.

Additional Resources

Related Guides

  • How to Change the Default SSH Port
  • SSH Key Management: Generation and Best Practices
  • Initial Security Configuration on Ubuntu/Debian
  • How to Configure Users and Permissions on Linux
  • Fail2Ban Configuration for Brute Force Protection