SaltStack Installation and Configuration

SaltStack is a powerful infrastructure automation and configuration management tool that uses a master-minion architecture for managing large-scale infrastructures. Salt combines the simplicity of configuration management with advanced remote execution capabilities, making it ideal for infrastructure orchestration. This guide covers master and minion installation, state management, pillar data, grains for targeting, formula organization, and the event-driven reactor.

Table of Contents

1. SaltStack Overview
2. Master Installation
3. Minion Installation and Configuration
4. Key Management
5. State Management
6. Pillar Data
7. Grains and Targeting
8. Formulas and Modules
9. Event-Driven Reactor
10. Conclusion

SaltStack Overview

SaltStack (or Salt) is an event-driven automation platform for infrastructure configuration and orchestration. Using a simple YAML syntax and a master-minion model, Salt enables fast, reliable configuration management and remote execution at scale.

Key components:

• Master: Central server managing minions and executing commands
• Minions: Remote systems managed by the master
• States: Declarative configuration definitions (YAML)
• Pillar: Private data store for each minion
• Grains: System information and custom data on minions
• Formulas: Reusable state modules and execution code
• Reactor: Event-driven automation

Architecture:

┌─────────────────────┐
│   Salt Master       │
│   (Central Control) │
└──────────┬──────────┘
           │
      ┌────┴────┬─────────┐
      │          │         │
    ZMQ       ZMQ       ZMQ
      │          │         │
      ▼          ▼         ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Minion 1 │ │ Minion 2 │ │ Minion 3 │
│ (Managed)│ │(Managed) │ │(Managed) │
└──────────┘ └──────────┘ └──────────┘

Master Installation

Install and configure the Salt Master.

Installation:

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y salt-master

# CentOS/RHEL
sudo yum install -y salt-master

# macOS
brew install saltstack

# Start master
sudo systemctl start salt-master
sudo systemctl enable salt-master

# Verify
sudo salt-key -L

Master configuration:

# /etc/salt/master
# Master IP address (interface to bind)
interface: 0.0.0.0

# Default port
port: 4506

# Minimum number of required minions for batch mode
min_open_files: 100000

# Key management
auto_accept: False  # Require manual key acceptance

# File roots - location of state files
file_roots:
  base:
    - /srv/salt/states
  dev:
    - /srv/salt/dev
  prod:
    - /srv/salt/prod

# Pillar roots - location of pillar data
pillar_roots:
  base:
    - /srv/salt/pillar

# Return jobs to this minion
return_minion: true

# Job cache settings
job_cache: True
job_cache_dir: /var/cache/salt/jobs

# Worker processes
worker_threads: 5

# Timeout for minion responses
timeout: 5

# Include custom configuration
include: /etc/salt/master.d/*.conf

Create directory structure:

# Salt file structure
sudo mkdir -p /srv/salt/states
sudo mkdir -p /srv/salt/dev
sudo mkdir -p /srv/salt/prod
sudo mkdir -p /srv/salt/pillar
sudo mkdir -p /etc/salt/master.d

# Set permissions
sudo chown -R salt:salt /srv/salt
sudo chown -R salt:salt /etc/salt

Test master connectivity:

# Ping all minions
sudo salt '*' test.ping

# Show minion details
sudo salt '*' grains.items

# Execute command on all minions
sudo salt '*' cmd.run "hostname"

Minion Installation and Configuration

Install and configure Salt minions.

Installation:

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y salt-minion

# CentOS/RHEL
sudo yum install -y salt-minion

# macOS
brew install salt-minion

# Start minion
sudo systemctl start salt-minion
sudo systemctl enable salt-minion

Minion configuration:

# /etc/salt/minion
# Master IP or hostname
master: salt-master.example.com

# Minion ID (defaults to hostname)
id: webserver-01

# Port for minion connection
port: 4505

# Connection timeout
auth_timeout: 5

# Authentication retries
auth_tries: 5

# Authentication wait
auth_wait: 3

# Use TCP keepalive
tcp_keepalive: True

# File roots mapping (optional)
file_client: remote

# Module refresh interval
module_refresh_interval: 60

# Custom grains
grains:
  environment: production
  role: webserver
  datacenter: us-west-1

# Logging
log_level: info
log_level_logfile: warning

# Capabilities
enable_fqdns_grains: True

# Include custom configuration
include: /etc/salt/minion.d/*.conf

Minion test:

# Test minion status
sudo salt-minion -d

# Check minion logs
sudo tail -f /var/log/salt/minion

# Test connectivity to master
sudo salt-call test.ping

Multiple masters configuration:

# /etc/salt/minion
# List of masters for failover
master:
  - master1.example.com
  - master2.example.com
  - master3.example.com

# Master discovery
master_type: failover

# Try all masters before failing
master_tries: -1

Key Management

Manage authentication keys between master and minions.

Accept minion keys:

# List unsigned keys
sudo salt-key -L

# Accept specific minion
sudo salt-key -a minion-id

# Accept all minions (use with caution)
sudo salt-key -A -y

# Reject minion
sudo salt-key -r minion-id

# Delete key
sudo salt-key -d minion-id

# List accepted keys
sudo salt-key -l accepted

# List rejected keys
sudo salt-key -l rejected

# List pending keys
sudo salt-key -l pending

Auto-accept minion keys:

# /etc/salt/master - Enable auto-accept (not recommended for production)
auto_accept: True

# Or use fingerprinting
auto_accept_fingerprints: True

Verify key fingerprints:

# Get minion fingerprint
sudo salt-call key.finger

# Get master fingerprint
sudo salt-key -F

# Verify minion fingerprint matches
sudo salt-key -f minion-id

State Management

Define infrastructure state using YAML.

Simple state file:

# /srv/salt/states/packages.sls
# Install packages
common-packages:
  pkg.installed:
    - names:
      - curl
      - wget
      - git
      - htop

# Ensure service running
nginx:
  pkg.installed:
    - name: nginx
  service.running:
    - name: nginx
    - enable: True
    - watch:
      - pkg: nginx

Apply state:

# Apply specific state
sudo salt '*' state.apply packages

# Apply to specific minions
sudo salt 'webserver-*' state.apply packages

# Apply multiple states
sudo salt '*' state.apply packages,nginx

# Test state (dry-run)
sudo salt '*' state.apply packages test=True

# Show what would change
sudo salt '*' state.apply packages --test

Complex state with templates:

# /srv/salt/states/nginx/init.sls
nginx-package:
  pkg.installed:
    - name: nginx

nginx-service:
  service.running:
    - name: nginx
    - enable: True
    - require:
      - pkg: nginx-package

nginx-config:
  file.managed:
    - name: /etc/nginx/nginx.conf
    - source: salt://nginx/files/nginx.conf
    - template: jinja
    - user: root
    - group: root
    - mode: 644
    - require:
      - pkg: nginx-package
    - watch_in:
      - service: nginx-service

nginx-vhost:
  file.managed:
    - name: /etc/nginx/sites-available/default
    - source: salt://nginx/files/vhost.conf.j2
    - template: jinja
    - context:
        server_name: {{ salt['grains.get']('fqdn') }}
        port: {{ pillar.get('nginx:port', 80) }}
    - require:
      - pkg: nginx-package
    - watch_in:
      - service: nginx-service

Template with Jinja:

# /srv/salt/nginx/files/vhost.conf.j2
server {
    listen {{ port }};
    server_name {{ server_name }};

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
    }
}

Pillar Data

Store sensitive and variable data per minion.

Pillar structure:

# /srv/salt/pillar/top.sls
base:
  '*':
    - common
  'webserver-*':
    - webserver
  'environment:production':
    - match: grain
    - production

# /srv/salt/pillar/common.sls
common:
  ntp_servers:
    - 0.ubuntu.pool.ntp.org
    - 1.ubuntu.pool.ntp.org

# /srv/salt/pillar/webserver.sls
nginx:
  port: 80
  worker_processes: 4
  upstream: 127.0.0.1:3000

app:
  name: myapp
  version: 1.0.0

# /srv/salt/pillar/production.sls
database:
  host: db.prod.internal
  port: 5432
  password: secure_password_here

Use pillar in states:

# /srv/salt/states/app.sls
nginx-config:
  file.managed:
    - name: /etc/nginx/sites-available/default
    - contents: |
        server {
            listen {{ pillar['nginx']['port'] }};
            location / {
                proxy_pass http://{{ pillar['app']['upstream'] }};
            }
        }

app-env:
  file.managed:
    - name: /opt/app/.env
    - contents: |
        APP_NAME={{ pillar['app']['name'] }}
        APP_VERSION={{ pillar['app']['version'] }}
        DB_HOST={{ pillar['database']['host'] }}
        DB_PORT={{ pillar['database']['port'] }}

Access pillar from minion:

# View all pillar data
sudo salt-call pillar.items

# View specific pillar value
sudo salt-call pillar.get nginx:port

# View with default
sudo salt-call pillar.get 'nginx:port' 80

Grains and Targeting

Use grains for system information and targeting.

View grains:

# List all grains
sudo salt '*' grains.items

# Get specific grain
sudo salt '*' grains.get os

# Get multiple grains
sudo salt '*' grains.items

Common grains:

os              Operating system
osrelease       OS version
osfullname      Full OS name
hostname        System hostname
fqdn            Fully qualified domain name
ipv4            IPv4 addresses
ipv6            IPv6 addresses
cpu_cores       Number of CPU cores
mem_total       Total memory
virtual         Virtualization type

Custom grains:

# /etc/salt/minion.d/custom_grains.conf
grains:
  environment: production
  role: webserver
  datacenter: us-west-1
  team: platform
  cost_center: engineering

Or define in state:

# /srv/salt/states/grains.sls
set-custom-grains:
  grains.present:
    - name: environment
    - value: production

set-role-grain:
  grains.present:
    - name: role
    - value: webserver

Targeting with grains:

# Target by OS
sudo salt -G 'os:Ubuntu' state.apply

# Target by role
sudo salt -G 'role:webserver' state.apply packages

# Target by multiple grains
sudo salt -G 'os:Ubuntu and role:webserver' state.apply

# List matching minions
sudo salt -G 'role:webserver' test.ping

# Compound targeting
sudo salt -C 'G@role:webserver and not G@environment:dev' state.apply

Formulas and Modules

Organize reusable configurations as formulas.

Formula structure:

nginx-formula/
├── nginx/
│   ├── init.sls
│   ├── files/
│   │   ├── nginx.conf
│   │   └── vhost.conf.j2
│   ├── defaults.yaml
│   └── map.jinja
├── pillar.example
└── README.md

Reusable formula:

# /srv/salt/nginx/init.sls
{% from "nginx/map.jinja" import nginx_settings with context %}

nginx-package:
  pkg.installed:
    - name: {{ nginx_settings.package }}

nginx-service:
  service.running:
    - name: {{ nginx_settings.service }}
    - enable: True

nginx-config:
  file.managed:
    - name: {{ nginx_settings.config_path }}
    - user: {{ nginx_settings.user }}
    - group: {{ nginx_settings.group }}
    - mode: 644
    - template: jinja
    - context:
        settings: {{ nginx_settings }}

Map file for OS abstraction:

# /srv/salt/nginx/map.jinja
{% set nginx_settings = salt['grains.filter_by']({
    'Debian': {
        'package': 'nginx',
        'service': 'nginx',
        'config_path': '/etc/nginx/nginx.conf',
        'user': 'www-data',
        'group': 'www-data',
    },
    'RedHat': {
        'package': 'nginx',
        'service': 'nginx',
        'config_path': '/etc/nginx/nginx.conf',
        'user': 'nginx',
        'group': 'nginx',
    },
}, merge=salt['pillar.get']('nginx:settings')) %}

Event-Driven Reactor

Automate responses to Salt events.

Reactor configuration:

# /etc/salt/master.d/reactor.conf
reactor:
  - 'salt/minion/*/start':
    - /srv/salt/reactor/minion_start.sls
  - 'salt/job/*/ret/*':
    - /srv/salt/reactor/job_return.sls

Reactor state:

# /srv/salt/reactor/minion_start.sls
# Triggered when minion starts
highstate_on_start:
  local.state.apply:
    - tgt: {{ data['id'] }}
    - arg:
      - highstate

notify_admin:
  local.cmd.run:
    - tgt: salt-master
    - arg:
      - 'echo "Minion {{ data[id] }} joined" | mail [email protected]'

Job return reactor:

# /srv/salt/reactor/job_return.sls
# Triggered when job completes
check_status:
  local.cmd.run:
    - tgt: salt-master
    - arg:
      - 'logger "Job {{ data[jid] }} completed on {{ data[id] }}"'

store_return:
  local.event.fire:
    - name: custom/job/complete
    - data:
        minion: {{ data['id'] }}
        jid: {{ data['jid'] }}

Fire custom events:

# Fire event from master
sudo salt-call event.fire_master "message" "tag"

# From minion
sudo salt-call event.fire "message" "custom/event/name"

# Async execution tracked by reactor
sudo salt '*' cmd.run 'sleep 10' -b 10  # Batch mode

Conclusion

SaltStack provides powerful infrastructure automation through a clean master-minion model and YAML-based configuration. By mastering state management, pillar data for configuration, grains for targeting, and the reactor for event-driven automation, you create scalable infrastructure management that grows with your organization. SaltStack's combination of configuration management and remote execution capabilities makes it ideal for complex infrastructure orchestration at any scale.