Netdata Installation for Real-Time Monitoring

Netdata is a real-time performance monitoring platform that automatically detects and collects thousands of system metrics with zero configuration. It provides real-time dashboards, anomaly detection, and can stream data to parent nodes for centralized monitoring. This guide covers installation, auto-detection, alarm configuration, and Cloud integration.

Table of Contents

• Introduction
• System Requirements
• Installation
• Initial Configuration
• Auto-Detection
• Alarms and Notifications
• Parent-Child Streaming
• Netdata Cloud Integration
• Custom Metrics
• Performance Tuning
• Troubleshooting
• Conclusion

Introduction

Netdata automatically detects and monitors everything on your system with beautiful, real-time dashboards. Unlike other monitoring tools requiring configuration, Netdata works out of the box while remaining highly configurable. It collects hundreds of system and application metrics from CPU, memory, disk, network, processes, and integrations with databases, web servers, and other services.

System Requirements

• Supported Linux systems: Ubuntu, Debian, CentOS, RHEL, Fedora, Alpine
• Minimum 50MB disk space
• Minimal CPU/memory overhead
• Internet connectivity for Cloud integration (optional)
• Root or sudo access for installation
• Recommended: 2GB RAM for parent nodes in clustered setups

Installation

Step 1: Automatic Installation

# Download and run installer
wget -O get-netdata.sh https://get.netdata.cloud/kickstart.sh
sh get-netdata.sh --stable-channel

# Or use Git installation
git clone https://github.com/netdata/netdata.git --depth=100
cd netdata
./netdata-installer.sh --stable-channel

Step 2: Manual Installation from Source

# Install dependencies (Ubuntu/Debian)
sudo apt-get update
sudo apt-get install -y \
  autoconf \
  autoconf-archive \
  autogen \
  automake \
  cmake \
  curl \
  distutils-python3 \
  gcc \
  git \
  libcap2-bin \
  libcurl4-openssl-dev \
  libssl-dev \
  libuv1-dev \
  libyaml-dev \
  libzstd-dev \
  make \
  mysql-client \
  pkg-config \
  python3 \
  python3-distutils \
  python3-pip \
  uuid-dev \
  zlib1g-dev

# Clone and build
git clone https://github.com/netdata/netdata.git --depth=100
cd netdata
./netdata-installer.sh --stable-channel --disable-go

# Start service
sudo systemctl enable netdata
sudo systemctl start netdata

Step 3: Verify Installation

# Check service status
sudo systemctl status netdata

# Verify port 19999 is listening
ss -tlnp | grep netdata

# Access Netdata dashboard
# Navigate to http://localhost:19999

Initial Configuration

Main Configuration File

Edit /etc/netdata/netdata.conf:

sudo nano /etc/netdata/netdata.conf

Key configuration sections:

[global]
  # Database
  memory mode = dbengine
  disk space = 256

  # Performance
  update every = 1
  run as user = netdata

[web]
  # Listen settings
  mode = static-threaded
  listen backlog = 4096
  default port = 19999
  
  # SSL configuration
  ssl certificate = /etc/netdata/ssl/cert.pem
  ssl key = /etc/netdata/ssl/key.pem

[plugins]
  # Enable/disable data collectors
  go.d plugin = yes
  python.d plugin = yes
  chart.d plugin = yes
  proc plugin = yes
  diskspace plugin = yes
  cgroup plugin = yes

Enable DBEngine Storage

For long-term data retention:

[global]
  memory mode = dbengine
  disk space = 1024  # MB for history
  dbengine update every = 60
  dbengine tier 0 = 60
  dbengine tier 1 = 600
  dbengine tier 2 = 3600

Auto-Detection

Automatic Plugin Discovery

Netdata automatically detects:

# View auto-detected modules
sudo ls -la /etc/netdata/go.d.conf/

# Available collectors
sudo netdata -W plugins

# Check running collectors
curl -s http://localhost:19999/api/v1/info | jq '.collectors'

Common Auto-Detected Services

- Apache/Nginx web servers
- MySQL/PostgreSQL databases
- Redis cache
- Docker containers
- Systemd services
- Process monitoring
- Network interfaces
- Disk I/O
- CPU metrics
- Memory metrics

Manual Plugin Configuration

# Enable/disable specific plugins
sudo nano /etc/netdata/go.d.conf

# Edit individual plugin configs
sudo nano /etc/netdata/go.d/mysql.conf
sudo nano /etc/netdata/go.d/postgres.conf
sudo nano /etc/netdata/go.d/redis.conf

Example MySQL configuration:

jobs:
  - name: mysql-local
    host: 127.0.0.1
    port: 3306
    username: netdata
    password: password
    ssl: no

Alarms and Notifications

Alarm Configuration

Edit /etc/netdata/health.d/ for health checks:

# Example: Monitor CPU
sudo nano /etc/netdata/health.d/cpu.conf

TEMPLATE: cpu_usage
  on: system.cpu
  class: System
  component: CPU
  type: CPU Usage
  lookup: average -10m percentage of cpu usage
  units: %
  every: 1m
  warn: $this > 80
  crit: $this > 95
  info: CPU usage is above normal
  to: sysadmin

Create Custom Alarms

sudo tee /etc/netdata/health.d/custom-alarms.conf > /dev/null << 'EOF'
# Monitor disk space
TEMPLATE: disk_usage
  on: disk.space
  class: Disk
  component: Disk Space
  lookup: average -1m percentage of used disk space
  units: %
  every: 1m
  warn: $this > 80
  crit: $this > 95

# Monitor memory
TEMPLATE: memory_usage
  on: system.ram
  class: Memory
  component: Memory
  lookup: average -1m percentage of used memory
  units: %
  every: 1m
  warn: $this > 85
  crit: $this > 95

# Monitor swap
TEMPLATE: swap_usage
  on: system.swap
  class: Memory
  component: Swap
  lookup: average -1m percentage of used swap
  units: %
  every: 1m
  warn: $this > 50
  crit: $this > 80
EOF

sudo systemctl restart netdata

Configure Notifications

Edit /etc/netdata/health_alarm_notify.conf:

sudo nano /etc/netdata/health_alarm_notify.conf

# Email notifications
SEND_EMAIL="YES"
EMAIL_SMTP_SERVER="smtp.gmail.com"
EMAIL_SMTP_PORT="587"
EMAIL_SMTP_ENCRYPTION="starttls"
EMAIL_SMTP_USERNAME="[email protected]"
EMAIL_SMTP_PASSWORD="app-password"
EMAIL_FROM="[email protected]"
DEFAULT_RECIPIENT_EMAIL="[email protected]"

# Slack notifications
SEND_SLACK="YES"
SLACK_WEBHOOK_URL="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
DEFAULT_RECIPIENT_SLACK="@devops"

# Discord
SEND_DISCORD="YES"
DISCORD_WEBHOOK_URL="https://discord.com/api/webhooks/YOUR/WEBHOOK"

# Custom webhook
SEND_CUSTOM="YES"
CUSTOM_WEBHOOK_URL="https://your-api.example.com/alerts"

Test Notifications

# Test email
sudo /opt/netdata/usr/libexec/netdata/plugins.d/send_email.sh \
  "EMAIL" "[email protected]" "Test from Netdata" \
  "This is a test notification"

# Test Slack
curl -X POST https://hooks.slack.com/services/YOUR/WEBHOOK \
  -H 'Content-type: application/json' \
  -d '{"text":"Test from Netdata"}'

Parent-Child Streaming

Configure Parent Node (Collector)

# Edit netdata.conf
sudo nano /etc/netdata/netdata.conf

[web]
  bind to = 0.0.0.0:19999

[stream]
  enabled = yes
  default port = 19999
  default history = 3600
  default mode = bind
  
  [API_KEY]
    allow from = 192.168.1.0/24
    default history = 3600
    default mode = bind
    hostname = Primary Parent Node

Configure Child Node (Forwarder)

# Edit child netdata.conf
sudo nano /etc/netdata/netdata.conf

[stream]
  enabled = yes
  destination = 192.168.1.10:19999
  api key = your-api-key-here
  timeout connect = 5
  timeout handshake = 20
  default port = 19999

Verify Streaming

# Check parent sees children
curl http://localhost:19999/api/v1/info | jq '.nodes'

# Monitor streaming bandwidth
curl http://localhost:19999/api/v1/data | grep -i stream

Netdata Cloud Integration

Connect to Netdata Cloud

# Install Cloud agent
wget -O /tmp/nc-agent.sh https://api.netdata.cloud/agent-install.sh
sudo bash /tmp/nc-agent.sh

# Or claim existing installation
sudo netdata-claim.sh -token=YOUR_TOKEN \
  -rooms=ROOM_ID \
  -url=https://app.netdata.cloud

Key Features

1. Centralized dashboard across multiple nodes
2. Anomaly detection and insights
3. Alert management and routing
4. Integration with webhooks
5. Historical data retention

Configuration

# View claim status
sudo cat /etc/netdata/.cloud.conf

# Disable/enable Cloud
sudo nano /etc/netdata/netdata.conf

[cloud]
  enabled = yes
  cloud base url = https://app.netdata.cloud

Custom Metrics

External Plugins

Create custom data collection:

sudo tee /usr/libexec/netdata/plugins.d/custom_metric.sh > /dev/null << 'EOF'
#!/bin/bash

# Netdata plugin format
source /usr/libexec/netdata/plugin_utils.sh

update_every=10

declare -A metrics

send_header_first "custom" "1" "$update_every" "incremental"

while sleep "$update_every"; do
  # Get custom metrics
  value1=$(your_command_here)
  value2=$(another_command)
  
  # Send to Netdata
  send_data_single "custom.metric1" "$value1"
  send_data_single "custom.metric2" "$value2"
done
EOF

chmod +x /usr/libexec/netdata/plugins.d/custom_metric.sh

Python Plugins

sudo tee /usr/libexec/netdata/plugins.d/custom_app.py > /dev/null << 'EOF'
#!/usr/bin/env python3

import json
import time
import sys

# Configuration
update_every = 10

def get_metrics():
    # Your custom logic here
    return {
        'custom.metric1': 100,
        'custom.metric2': 200
    }

print("CHART custom.metrics '' 'Custom Metrics' 'units' 'custom' '' line", flush=True)
print("DIMENSION metric1 '' incremental 1 1", flush=True)
print("DIMENSION metric2 '' incremental 1 1", flush=True)

while True:
    metrics = get_metrics()
    print(f"BEGIN custom.metrics {int(time.time())}", flush=True)
    
    for key, value in metrics.items():
        print(f"SET {key.split('.')[1]} = {value}", flush=True)
    
    print("END", flush=True)
    time.sleep(update_every)
EOF

chmod +x /usr/libexec/netdata/plugins.d/custom_app.py

Performance Tuning

Memory Management

[global]
  # Reduce memory for constrained systems
  memory mode = save
  
  # Or use dbengine with size limits
  memory mode = dbengine
  disk space = 128  # Limited disk

[plugins]
  # Disable unnecessary plugins
  python.d plugin = no
  chart.d plugin = no

Database Tuning

[global]
  # Sampling
  update every = 2  # Increase sample interval
  
  # Storage
  dbengine multihost-db = yes
  
  # History tiers
  dbengine tier 0 = 3600  # 1 hour
  dbengine tier 1 = 86400  # 1 day

Network Optimization

[web]
  # Compression
  enable gzip = yes
  gzip compression level = 6
  gzip minimum size = 1024
  
  # Caching
  enable web response gzip = yes
  max connections = 512

Troubleshooting

Check Service Status

# Service status
sudo systemctl status netdata

# View logs
sudo journalctl -u netdata -f

# Check process
ps aux | grep netdata

Verify Configuration

# Check enabled collectors
sudo netdata -W plugins

# Validate configuration
sudo netdata -W config

# Check dashboard
curl -s http://localhost:19999/api/v1/info | jq .

Debug Plugins

# Run plugin directly
sudo /opt/netdata/usr/libexec/netdata/plugins.d/go.d.plugin version

# Check plugin logs
sudo tail -f /var/log/netdata/go.d.log

# Test specific collector
sudo /opt/netdata/usr/libexec/netdata/plugins.d/go.d.plugin \
  debug -m mysql

Performance Issues

# Monitor Netdata itself
curl http://localhost:19999/api/v1/data?chart=netdata.agent_cpu

# Check memory usage
curl http://localhost:19999/api/v1/data?chart=netdata.agent_memory

# Database performance
curl http://localhost:19999/api/v1/data?chart=netdata.dbengine_io

Conclusion

Netdata provides real-time system monitoring with zero configuration overhead. By following this guide, you've deployed a powerful monitoring platform that automatically detects system metrics and services. Focus on customizing alarms for your specific infrastructure, leveraging Cloud integration for centralized monitoring, and creating dashboards that highlight critical metrics. The combination of automatic detection, flexible configuration, and beautiful visualizations makes Netdata an excellent choice for infrastructure monitoring.