Falco Runtime Security for Containers
Falco is a cloud-native runtime security tool that detects anomalous behavior in containers and Kubernetes by monitoring system calls, alerting on policy violations in real time. This guide covers deploying Falco on Linux and Kubernetes, customizing detection rules, configuring Slack and webhook alerts, and building incident response workflows.
Prerequisites
- Ubuntu 22.04/Debian 12 or CentOS/Rocky 9
- Linux kernel 5.4+ (for eBPF driver) or kernel headers installed (for kmod driver)
- Root access (Falco requires kernel-level access)
- Docker or Kubernetes for container monitoring
- For Kubernetes:
helminstalled
Install Falco on Linux
# Ubuntu/Debian
curl -fsSL https://falco.org/repo/falcosecurity-packages.asc | \
sudo gpg --dearmor -o /usr/share/keyrings/falco-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/falco-archive-keyring.gpg] \
https://download.falco.org/packages/deb stable main" | \
sudo tee /etc/apt/sources.list.d/falcosecurity.list
sudo apt update && sudo apt install -y falco
# CentOS/Rocky
sudo tee /etc/yum.repos.d/falcosecurity.repo << 'EOF'
[falcosecurity]
name=falcosecurity repository
baseurl=https://download.falco.org/packages/rpm/stable/$basearch/
enabled=1
gpgcheck=1
gpgkey=https://falco.org/repo/falcosecurity-packages.asc
EOF
sudo dnf install -y falco
# The installer will ask which driver to use:
# - Modern eBPF (recommended for kernel 5.8+)
# - eBPF (kernel 4.14+ with eBPF support)
# - Kernel module (kmod) - fallback for older kernels
# Enable and start Falco
sudo systemctl enable --now falco
# Verify Falco is running
sudo systemctl status falco
sudo falco --version
Install Falco in Kubernetes
Deploy Falco using Helm:
# Add the Falco Helm repository
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update
# Install Falco with modern eBPF driver (recommended)
helm install falco falcosecurity/falco \
--namespace falco \
--create-namespace \
--set driver.kind=modern_ebpf \
--set tty=true
# Install with custom values file
cat > falco-values.yaml << 'EOF'
driver:
kind: modern_ebpf
falco:
grpc:
enabled: true
grpc_output:
enabled: true
json_output: true
log_stderr: true
log_syslog: false
loglevel: info
falcosidekick:
enabled: true
webui:
enabled: true
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
operator: Exists
EOF
helm install falco falcosecurity/falco \
--namespace falco \
--create-namespace \
-f falco-values.yaml
# Check deployment
kubectl -n falco get pods
kubectl -n falco logs -l app.kubernetes.io/name=falco --tail=50
Understanding and Customizing Rules
Falco rules define what constitutes suspicious behavior. Default rules are in /etc/falco/falco_rules.yaml.
View existing rules:
# List all loaded rules
sudo falco --list
# Check what files Falco loads
sudo falco --config /etc/falco/falco.yaml --validate /etc/falco/falco_rules.yaml
Create custom rules in /etc/falco/falco_rules.local.yaml (override or add to defaults):
# /etc/falco/falco_rules.local.yaml
# Override the default rule to reduce noise (append to the condition)
- rule: Terminal shell in container
append: true
condition: and not container.image.repository = "my-debug-image"
# Custom rule: Alert on sensitive file reads
- rule: Read sensitive files in container
desc: Detect reads of sensitive files inside containers
condition: >
open_read and container and
(fd.name startswith /etc/shadow or
fd.name startswith /etc/sudoers or
fd.name startswith /root/.ssh or
fd.name startswith /etc/kubernetes/admin.conf)
output: >
Sensitive file opened for reading in container
(user=%user.name user_uid=%user.uid command=%proc.cmdline
file=%fd.name container_id=%container.id image=%container.image.repository)
priority: WARNING
tags: [container, filesystem, mitre_credential_access]
# Custom rule: Alert on outbound connections from database containers
- rule: Unexpected outbound connection from database
desc: Database container making outbound network connection
condition: >
outbound and container and
container.image.repository contains "postgres" and
not fd.sip in (allowed_db_destinations)
output: >
Unexpected network connection from database container
(user=%user.name command=%proc.cmdline connection=%fd.name
container_id=%container.id image=%container.image.repository)
priority: CRITICAL
tags: [network, database]
# Define a macro for reuse
- macro: allowed_db_destinations
condition: (fd.sip="10.0.0.100" or fd.sip="10.0.0.101")
Reload rules without restarting:
sudo kill -1 $(pidof falco) # Send SIGHUP to reload
# Or:
sudo systemctl reload falco
Configure Alerts and Notifications
Falco outputs alerts to multiple destinations. Configure in /etc/falco/falco.yaml:
sudo nano /etc/falco/falco.yaml
# Output channels in falco.yaml
# Standard output (stdout)
stdout_output:
enabled: true
# Syslog output
syslog_output:
enabled: true
# File output (JSON format for log aggregation)
file_output:
enabled: true
keep_alive: false
filename: /var/log/falco/falco_events.log
# JSON output format
json_output: true
json_include_output_property: true
# Program output (pipe to a script or falcosidekick)
program_output:
enabled: true
keep_alive: false
program: "jq '{summary: .output, priority: .priority, time: .time}' | curl -s -X POST -H 'Content-Type: application/json' -d @- https://webhook.example.com/falco"
Slack and Webhook Integration
Use falcosidekick for rich notification routing:
# Install falcosidekick
curl -L https://github.com/falcosecurity/falcosidekick/releases/latest/download/falcosidekick_linux_amd64.tar.gz \
-o /tmp/falcosidekick.tar.gz
tar xvf /tmp/falcosidekick.tar.gz -C /usr/local/bin/ falcosidekick
# Configure falcosidekick
sudo tee /etc/falcosidekick/config.yaml << 'EOF'
listenaddress: "0.0.0.0"
listenport: 2801
debug: false
slack:
webhookurl: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"
channel: "#security-alerts"
username: "Falco"
icon: "https://falco.org/img/brand/falco-logo.png"
minimumpriority: "warning"
messageformat: "Rule: *{{.Rule}}* triggered by {{.OutputFields.container_image}} (Priority: *{{.Priority}}*)"
# PagerDuty for critical alerts
pagerduty:
apikey: "YOUR_PAGERDUTY_API_KEY"
routingkey: "YOUR_ROUTING_KEY"
minimumpriority: "critical"
# Generic webhook
webhook:
address: "https://webhook.example.com/falco-events"
method: "POST"
minimumpriority: "notice"
EOF
# Create systemd service for falcosidekick
sudo tee /etc/systemd/system/falcosidekick.service << 'EOF'
[Unit]
Description=Falco Sidekick notification forwarder
After=network.target
[Service]
ExecStart=/usr/local/bin/falcosidekick -c /etc/falcosidekick/config.yaml
Restart=on-failure
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl enable --now falcosidekick
Configure Falco to send events to falcosidekick:
# In falco.yaml, add:
program_output:
enabled: true
keep_alive: true
program: "curl -d @- localhost:2801/"
Testing Falco Rules
Generate test events to verify rules are working:
# Test rule: shell spawned inside a container
docker run -it --rm ubuntu:22.04 bash -c "echo 'testing falco'"
# Test rule: sensitive file access
docker run --rm ubuntu:22.04 cat /etc/shadow
# Test rule: privileged container started
docker run --rm --privileged alpine id
# Test rule: package manager run in container
docker run --rm ubuntu:22.04 apt-get install -y curl
# Watch Falco alerts in real-time
sudo journalctl -fu falco
# Or:
sudo tail -f /var/log/falco/falco_events.log | jq '.'
Use event-generator to simulate security events:
# Install the Falco event generator
docker run -it --rm falcosecurity/event-generator run syscall
# Run a specific test
docker run -it --rm falcosecurity/event-generator run syscall.ReadSensitiveFileUntrusted
Incident Response Workflows
Automate responses to specific Falco alerts:
# Automated response script - isolate a compromised container
cat > /usr/local/bin/falco-respond.sh << 'EOF'
#!/bin/bash
# Called by Falco's program_output or falcosidekick
# Read event from stdin
EVENT=$(cat)
RULE=$(echo "${EVENT}" | jq -r '.rule')
PRIORITY=$(echo "${EVENT}" | jq -r '.priority')
CONTAINER_ID=$(echo "${EVENT}" | jq -r '.output_fields.container_id')
# Log the event
echo "$(date): Falco alert - Rule: ${RULE}, Priority: ${PRIORITY}, Container: ${CONTAINER_ID}" \
>> /var/log/falco-response.log
# Respond to critical events
if [[ "${PRIORITY}" == "CRITICAL" ]] && [[ -n "${CONTAINER_ID}" ]]; then
# Pause the container (stops new processes, preserves state for forensics)
docker pause "${CONTAINER_ID}" 2>/dev/null
echo "$(date): Paused container ${CONTAINER_ID} due to critical Falco alert: ${RULE}" \
>> /var/log/falco-response.log
# Capture container state for forensics
docker inspect "${CONTAINER_ID}" > "/var/log/forensics-${CONTAINER_ID}.json" 2>/dev/null
fi
EOF
chmod +x /usr/local/bin/falco-respond.sh
For Kubernetes, use falcosidekick with kubeless or a webhook to automatically quarantine pods:
# falcosidekick Kubernetes config
kubeless:
namespace: "default"
function: "isolate-pod"
minimumpriority: "critical"
# Or use a Kubernetes NetworkPolicy to isolate the pod
Troubleshooting
Falco not detecting events:
# Check Falco is loaded and driver is working
sudo systemctl status falco
sudo falco --list | head -20
# Verify the kernel driver is loaded
lsmod | grep falco
# Test with a known-bad action
docker run --rm ubuntu:22.04 cat /etc/shadow
sudo journalctl -u falco -n 20 --no-pager
Rule syntax errors:
# Validate rule files
sudo falco --config /etc/falco/falco.yaml \
--validate /etc/falco/falco_rules.local.yaml
# Check for YAML syntax errors
python3 -c "import yaml; yaml.safe_load(open('/etc/falco/falco_rules.local.yaml'))"
High CPU usage:
# Check which rules are generating the most events
sudo falco --stats_interval=10 2>&1 | grep -i "rule"
# Suppress noisy rules by adding conditions
# Example: ignore specific containers from a noisy rule
- rule: Noisy Rule
append: true
condition: and not container.image.repository = "known-noisy-image"
Falco not starting after kernel update:
# Rebuild the kernel module for the new kernel
sudo dkms install -m falco -v $(falco --version | grep -o '[0-9.]*')
# Or switch to eBPF driver (no recompilation needed)
sudo apt install -y falco-driver-loader
sudo falco-driver-loader modern_ebpf
Conclusion
Falco provides real-time threat detection for containers and Kubernetes by monitoring system calls at the kernel level, catching shell spawns, privilege escalations, and sensitive file accesses that static scanning misses. Custom rules let you tailor detection to your specific environment, while falcosidekick routes alerts to Slack, PagerDuty, or automated response scripts. The key to effective Falco deployment is starting with the default rules, suppressing false positives incrementally, and building automated responses for the highest-priority alerts.


