Nebula Overlay Network Installation

Nebula is a scalable overlay networking tool that creates secure mesh VPN connections between nodes using certificates and a lighthouse-based discovery system. Developed by Slack, Nebula uses its own lightweight tunneling protocol with built-in firewall rules, making it ideal for connecting VPS instances, bare-metal servers, and cloud resources across multiple data centers with group-based access control.

Prerequisites

  • Ubuntu 20.04+, Debian 11+, CentOS 8+, or Rocky Linux 8+
  • At least one node with a public IP address (lighthouse)
  • Root or sudo access on all nodes
  • UDP port 4242 open on the lighthouse node

Downloading Nebula

# Download the latest release (check https://github.com/slackhq/nebula/releases)
NEBULA_VERSION="v1.9.0"
curl -LO "https://github.com/slackhq/nebula/releases/download/${NEBULA_VERSION}/nebula-linux-amd64.tar.gz"

# Extract binaries
tar -xzf nebula-linux-amd64.tar.gz

# Install system-wide
sudo mv nebula nebula-cert /usr/local/bin/
sudo chmod +x /usr/local/bin/nebula /usr/local/bin/nebula-cert

# Verify installation
nebula --version
nebula-cert --version

Certificate Authority Setup

The CA is used to sign all node certificates. Keep the CA key secure — it should not be on any network node.

# Create a directory for CA files (do this on a secure admin machine)
mkdir -p ~/nebula-ca && cd ~/nebula-ca

# Generate the CA certificate and key
nebula-cert ca \
  --name "MyOrg Nebula CA" \
  --duration 8760h    # 1 year validity

# This creates:
# - ca.crt  (distribute to all nodes)
# - ca.key  (keep secret, never distribute)

ls -la
# ca.crt  ca.key

Generating Node Certificates

Generate a certificate for each node using the CA. Assign each node a unique IP within the Nebula subnet (e.g., 10.10.0.0/16).

# Generate certificate for the lighthouse node
nebula-cert sign \
  --name "lighthouse1" \
  --ip "10.10.0.1/16" \
  --ca-crt ca.crt \
  --ca-key ca.key

# Generate certificate for an application server
nebula-cert sign \
  --name "app-server1" \
  --ip "10.10.0.10/16" \
  --groups "servers,app" \
  --ca-crt ca.crt \
  --ca-key ca.key

# Generate certificate for a developer workstation
nebula-cert sign \
  --name "dev-laptop" \
  --ip "10.10.0.100/16" \
  --groups "developers" \
  --ca-crt ca.crt \
  --ca-key ca.key

# View certificate details
nebula-cert print --path app-server1.crt

Copy certificates to each node:

# Copy files to lighthouse node
scp ca.crt lighthouse1.crt lighthouse1.key user@lighthouse-public-ip:/etc/nebula/

# Copy files to app server
scp ca.crt app-server1.crt app-server1.key user@app-server:/etc/nebula/

Lighthouse Configuration

The lighthouse is a publicly reachable node that helps peers find each other. Create /etc/nebula/config.yml on the lighthouse:

sudo mkdir -p /etc/nebula

sudo tee /etc/nebula/config.yml <<'EOF'
pki:
  ca: /etc/nebula/ca.crt
  cert: /etc/nebula/lighthouse1.crt
  key: /etc/nebula/lighthouse1.key

static_host_map: {}   # Lighthouse has no upstream lighthouses

lighthouse:
  am_lighthouse: true
  interval: 60

listen:
  host: 0.0.0.0
  port: 4242

punchy:
  punch: true
  respond: true

tun:
  disabled: false
  dev: nebula1
  drop_local_broadcast: false
  drop_multicast: false

logging:
  level: info
  format: text

firewall:
  outbound:
    - port: any
      proto: any
      host: any

  inbound:
    - port: any
      proto: icmp
      host: any
EOF

Node Configuration

Create /etc/nebula/config.yml on all non-lighthouse nodes:

sudo mkdir -p /etc/nebula

sudo tee /etc/nebula/config.yml <<'EOF'
pki:
  ca: /etc/nebula/ca.crt
  cert: /etc/nebula/app-server1.crt
  key: /etc/nebula/app-server1.key

# Point to the lighthouse's public IP and Nebula port
static_host_map:
  "10.10.0.1": ["203.0.113.10:4242"]

lighthouse:
  am_lighthouse: false
  interval: 60
  hosts:
    - "10.10.0.1"

listen:
  host: 0.0.0.0
  port: 4242

punchy:
  punch: true
  respond: true

tun:
  disabled: false
  dev: nebula1

logging:
  level: info
  format: text

firewall:
  outbound:
    - port: any
      proto: any
      host: any

  inbound:
    - port: any
      proto: icmp
      host: any
    - port: 22
      proto: tcp
      groups:
        - developers
        - servers
    - port: 443
      proto: tcp
      groups:
        - servers
EOF

Firewall Rules and Group Access Control

Nebula's firewall uses group membership from certificates to control access. Groups are embedded in the certificate at signing time.

# Example: Allow only "developers" group to reach SSH on "servers" group
firewall:
  inbound:
    - port: 22
      proto: tcp
      groups:
        - developers

    # Allow servers to communicate with each other on any port
    - port: any
      proto: any
      groups:
        - servers

    # Allow monitoring tools to reach all nodes
    - port: 9100
      proto: tcp
      groups:
        - monitoring

    # Allow ICMP from anywhere in the overlay
    - port: any
      proto: icmp
      host: any

# Validate your config file before deploying
nebula --config /etc/nebula/config.yml --test

Running Nebula as a Service

# Create a systemd service unit
sudo tee /etc/systemd/system/nebula.service <<'EOF'
[Unit]
Description=Nebula Overlay Network
Wants=basic.target network-online.target
After=basic.target network.target network-online.target
Before=sshd.service

[Service]
SyslogIdentifier=nebula
ExecReload=/bin/kill -HUP $MAINPID
ExecStart=/usr/local/bin/nebula --config /etc/nebula/config.yml
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

# Enable and start the service
sudo systemctl daemon-reload
sudo systemctl enable --now nebula

# Check status
sudo systemctl status nebula
sudo journalctl -u nebula -f

Open the required firewall port:

# Ubuntu/Debian (UFW)
sudo ufw allow 4242/udp

# CentOS/Rocky (firewalld)
sudo firewall-cmd --add-port=4242/udp --permanent
sudo firewall-cmd --reload

Troubleshooting

Nodes cannot connect to lighthouse:

# Verify UDP 4242 is open on the lighthouse
sudo ss -ulnp | grep 4242

# Test UDP reachability from a node
nc -zu lighthouse-public-ip 4242

# Check Nebula logs for handshake errors
sudo journalctl -u nebula -n 50

Certificate validation errors:

# Verify certificate details
nebula-cert print --path /etc/nebula/app-server1.crt

# Check CA matches across nodes
nebula-cert verify --ca /etc/nebula/ca.crt --crt /etc/nebula/app-server1.crt

# Confirm system time is synchronized (certificates are time-sensitive)
timedatectl status

Nodes appear in lighthouse but cannot ping each other:

# Check firewall rules allow ICMP
# In config.yml, ensure this inbound rule exists:
# - port: any
#   proto: icmp
#   host: any

# Ping across the overlay network
ping 10.10.0.10

# Show current Nebula interface
ip addr show nebula1
ip route show dev nebula1

Conclusion

Nebula provides a highly scalable overlay network with a strong security model based on certificate authorities and group-based firewall policies, making it well-suited for multi-site infrastructure where fine-grained access control is required. By embedding group membership directly in certificates and enforcing rules at the kernel level, Nebula eliminates the need for separate firewall infrastructure while enabling zero-trust networking across cloud providers and bare-metal servers.