Chef Installation and Configuration

Chef is a flexible infrastructure automation platform using a client-server model with a Ruby-based domain-specific language for defining system configurations. Chef separates code (cookbooks) from data (attributes, data bags) and enables powerful, idempotent infrastructure management. This guide covers workstation setup, server installation, node bootstrapping, cookbook creation, recipe development, data bags, and kitchen testing.

Table of Contents

1. Chef Overview
2. Chef Server Installation
3. Workstation Setup
4. Node Bootstrap
5. Cookbook Structure
6. Writing Recipes
7. Attributes and Data Bags
8. Testing with Test Kitchen
9. Chef Automate Integration
10. Conclusion

Chef Overview

Chef is an infrastructure-as-code platform using a client-server architecture where a central Chef Server manages configurations deployed to Chef nodes. Chef uses Ruby for its domain-specific language, providing powerful programming capabilities for infrastructure automation.

Key components:

• Chef Server: Central authority managing node configurations
• Chef Workstation: Development environment for creating cookbooks
• Chef Client: Runs on managed nodes, pulls configuration from server
• Cookbooks: Collections of recipes, attributes, templates, and files
• Recipes: Ruby code defining infrastructure state
• Attributes: Variables and default configuration values
• Data Bags: Encrypted data storage for secrets and configurations

Architecture:

┌──────────────────────┐
│    Chef Server       │
│  (Central Authority) │
└──────────┬───────────┘
           │ API
     ┌─────┴─────┬──────────┐
     │            │          │
  Client        Client    Client
  (Ubuntu)     (CentOS)  (Debian)
     │            │          │
  Pull config  Pull config  Pull config
     │            │          │
  Converge    Converge    Converge
  (Apply)     (Apply)     (Apply)

Chef Server Installation

Install and configure Chef Server for managing infrastructure.

Installation:

# Download Chef Server
wget https://packages.chef.io/files/stable/chef-server/15.3.0/ubuntu/20.04/chef-server-core_15.3.0-1_amd64.deb

# Install
sudo dpkg -i chef-server-core_15.3.0-1_amd64.deb

# Reconfigure
sudo chef-server-ctl reconfigure

# Create admin user
sudo chef-server-ctl user-create admin Admin User [email protected] 'password' --filename admin.pem

# Create organization
sudo chef-server-ctl org-create myorg "My Organization" --association_user admin --filename myorg-validator.pem

# Enable Chef Manage (web UI)
sudo chef-server-ctl install chef-manage
sudo chef-server-ctl reconfigure

Access Chef Server:

# Web UI (requires SSL)
https://chef-server.example.com/organizations/myorg

# API endpoint
https://chef-server.example.com/organizations/myorg

Chef Server configuration:

# /etc/chef-server/chef-server.rb
# Server API port
nginx['port'] = 443

# Enable Chef Analytics
analytics_enabled = true

# Memory allocation
opscode_erchef['max_request_size'] = 2000000

# PostgreSQL settings
postgresql['shared_buffers'] = '256MB'

Workstation Setup

Set up Chef on development machines.

Install Chef Workstation:

# macOS
brew install chef-workstation

# Ubuntu/Debian
wget https://packages.chef.io/files/stable/chef-workstation/21.1.263/ubuntu/20.04/chef-workstation_21.1.263-1_amd64.deb
sudo dpkg -i chef-workstation_21.1.263-1_amd64.deb

# Windows
choco install chef-workstation

# Verify
chef -v
kitchen version
cookstyle --version

Configure knife:

# Create knife configuration
mkdir -p ~/.chef
cd ~/.chef

# Create knife.rb
cat > knife.rb << 'EOF'
current_dir = File.dirname(__FILE__)
log_level                :info
log_location             STDOUT
node_name                'admin'
client_key               "#{current_dir}/admin.pem"
validation_client_name   'myorg-validator'
validation_key           "#{current_dir}/myorg-validator.pem"
chef_server_url          'https://chef-server.example.com/organizations/myorg'
cookbook_path            ["#{current_dir}/../cookbooks"]
EOF

# Copy keys from Chef Server
scp admin@chef-server:admin.pem ~/.chef/
scp admin@chef-server:myorg-validator.pem ~/.chef/

# Test connection
knife client list

Create cookbook:

# Generate cookbook
chef generate cookbook my_app

# Structure created:
# my_app/
# ├── recipes/
# │   └── default.rb
# ├── attributes/
# │   └── default.rb
# ├── templates/
# ├── files/
# ├── spec/
# ├── test/
# ├── metadata.rb
# └── README.md

Node Bootstrap

Add nodes to Chef Server and configure management.

Bootstrap node:

# SSH bootstrap (most common)
knife bootstrap 192.168.1.100 \
  --ssh-user ubuntu \
  --sudo \
  --identity-file ~/.ssh/id_rsa \
  --node-name web01 \
  --run-list 'recipe[base],recipe[nginx]' \
  --chef-license accept

# Bootstrap with custom JSON
knife bootstrap 192.168.1.100 \
  --ssh-user ubuntu \
  --sudo \
  --node-name web02 \
  --json-attributes '{"nginx":{"port":8080}}'

# Windows bootstrap
knife bootstrap windows winrm 192.168.1.100 \
  --winrm-user Administrator \
  --winrm-password 'P@ssw0rd' \
  --node-name win-server

Manage nodes:

# List nodes
knife node list

# Show node details
knife node show web01

# Edit node
knife node edit web01

# Delete node
knife node delete web01

# Run Chef on node
knife ssh 'name:web01' 'sudo chef-client'

# SSH to node with knife
knife ssh 'name:web01' 'bash -l' -x ubuntu

Cookbook Structure

Organize Chef code in properly structured cookbooks.

Standard cookbook layout:

my_app/
├── metadata.rb              # Cookbook metadata
├── README.md                # Documentation
├── attributes/
│   └── default.rb           # Default attributes
├── recipes/
│   ├── default.rb           # Main recipe
│   ├── nginx.rb             # Nginx setup
│   └── database.rb          # Database setup
├── templates/
│   ├── nginx.conf.erb       # Nginx config template
│   └── app.conf.erb         # App config template
├── files/
│   ├── default/
│   │   └── app.jar          # Static files
│   └── ubuntu/              # OS-specific files
├── libraries/
│   └── helpers.rb           # Custom libraries
├── providers/
│   └── custom.rb            # Custom resources
├── resources/
│   └── custom.rb            # Custom resources
├── spec/
│   ├── spec_helper.rb
│   └── unit/
│       └── recipes/
│           └── default_spec.rb
├── test/
│   └── integration/
│       └── default/
│           └── default_test.rb
└── .kitchen.yml             # Test Kitchen config

metadata.rb:

# my_app/metadata.rb
name 'my_app'
description 'Application deployment cookbook'
version '1.0.0'
chef_version '>= 15.0'

maintainer 'DevOps Team'
maintainer_email '[email protected]'
license 'Apache-2.0'

supports 'ubuntu', '= 20.04'
supports 'centos', '>= 7.0'

depends 'ark', '~> 10.0'
depends 'nodejs', '~> 7.0'
depends 'postgresql', '~> 11.0'

Writing Recipes

Create recipes that define infrastructure state.

Basic recipe:

# my_app/recipes/default.rb
# Update package index
apt_update 'update'

# Install packages
package %w[curl wget git vim htop]

# Create directory
directory '/opt/app' do
  owner 'app'
  group 'app'
  mode '0755'
end

# Manage user
user 'app' do
  comment 'Application user'
  home '/home/app'
  shell '/bin/bash'
  manage_home true
end

# Create config file
file '/etc/app/config.yml' do
  content <<-EOU
  app:
    name: myapp
    version: 1.0.0
  EOU
  owner 'root'
  group 'root'
  mode '0644'
  action :create
end

Advanced recipe with templates and conditionals:

# my_app/recipes/nginx.rb
# Include attributes
include_recipe 'my_app::attributes'

# Install Nginx
package 'nginx'

# Create config from template
template '/etc/nginx/sites-available/default' do
  source 'nginx.conf.erb'
  owner 'root'
  group 'root'
  mode '0644'
  variables(
    server_name: node['app']['hostname'],
    app_port: node['app']['port'],
    upstream_servers: node['app']['servers']
  )
  notifies :restart, 'service[nginx]'
end

# Enable Nginx service
service 'nginx' do
  action [:enable, :start]
end

# Conditional installation
if node['platform'] == 'ubuntu'
  apt_package 'nginx-full'
elsif node['platform'] == 'centos'
  yum_package 'nginx'
end

# Execute command
execute 'configure-app' do
  command '/opt/app/setup.sh'
  not_if { ::File.exist?('/opt/app/.configured') }
  notifies :restart, 'service[myapp]'
end

# Ruby block for complex logic
ruby_block 'setup-database' do
  block do
    db_password = Chef::EncryptedDataBagItem.load('secrets', 'db')['password']
    # Setup database with password
  end
  only_if { node['app']['enable_db'] }
end

Attributes and Data Bags

Manage configuration data separately from code.

Attributes:

# my_app/attributes/default.rb
default['app']['name'] = 'myapp'
default['app']['version'] = '1.0.0'
default['app']['port'] = 3000

override['app']['hostname'] = 'app.example.com' if node.chef_environment == 'production'

# Platform-specific
case node['platform']
when 'ubuntu'
  default['app']['user'] = 'www-data'
when 'centos'
  default['app']['user'] = 'apache'
end

# Array attributes
default['app']['packages'] = %w[
  curl
  wget
  git
]

# Hash attributes
default['nginx']['sites'] = {
  'default' => {
    port: 80,
    root: '/var/www/html'
  }
}

Data bags for secrets:

# Create data bag
knife data bag create secrets

# Create encrypted item
knife data bag from file secrets <<'EOF'
{
  "id": "db",
  "password": "secure_password_here",
  "username": "admin"
}
EOF

Use data bag in recipe:

# my_app/recipes/database.rb
# Load encrypted data bag
db_secrets = Chef::EncryptedDataBagItem.load('secrets', 'db')

# Use in resource
execute 'setup-database' do
  command "mysql -u #{db_secrets['username']} -p#{db_secrets['password']}"
  sensitive true  # Don't log password
end

Testing with Test Kitchen

Test cookbooks before deploying to production.

Kitchen configuration:

# .kitchen.yml
---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  multiple_converge: 2
  enforce_idempotence: true
  deprecations_as_errors: true

verifier:
  name: inspec

platforms:
  - name: ubuntu-20.04
    driver:
      box: bento/ubuntu-20.04
      cpus: 2
      memory: 2048
  
  - name: centos-8
    driver:
      box: bento/centos-8

suites:
  - name: default
    run_list:
      - recipe[my_app::default]
    verifier:
      inspec_tests:
        - test/integration/default

  - name: nginx
    run_list:
      - recipe[my_app::default]
      - recipe[my_app::nginx]
    verifier:
      inspec_tests:
        - test/integration/nginx

Run Kitchen tests:

# Create test instances
kitchen create

# Converge (apply recipe)
kitchen converge

# Run verification (tests)
kitchen verify

# Test idempotence
kitchen converge again

# Destroy test instances
kitchen destroy

# All-in-one test
kitchen test

InSpec test example:

# test/integration/default/default_test.rb
describe package('curl') do
  it { should be_installed }
end

describe service('nginx') do
  it { should be_installed }
  it { should be_enabled }
  it { should be_running }
end

describe file('/opt/app') do
  it { should exist }
  its('owner') { should eq 'app' }
  its('group') { should eq 'app' }
end

Chef Automate Integration

Integrate with Chef Automate for compliance and visibility.

Chef Automate setup:

# Install Chef Automate
curl https://packages.chef.io/files/stable/automate/latest/chef-automate_linux_amd64.zip | unzip

sudo ./chef-automate deploy config.toml \
  --product automate \
  --product infra-server \
  --accept-terms-and-log-to-file

# Access dashboard
https://automate.example.com

Configure Chef client for Automate:

# /etc/chef/client.rb
# Send data to Chef Automate
data_collector_url 'https://automate.example.com/data-collector/v0/'
data_collector_token 'AUTOMATE_TOKEN'

# Enable policyfile
use_policyfile = true
policy_name 'prod'
policy_group 'production'

Policyfile for versioning:

# Policyfile.rb
name 'prod'
description 'Production infrastructure'
default_source :chef_server, 'https://chef-server.example.com'

run_list 'recipe[base]', 'recipe[nginx]'

cookbook 'base', '~> 1.0'
cookbook 'nginx', '~> 5.0'

Conclusion

Chef provides powerful infrastructure automation through flexible recipes, reusable cookbooks, and strong testing capabilities. By mastering recipe development, attribute management, data bags for secrets, and Kitchen for testing, you can create reliable, maintainable infrastructure code. Integration with Chef Automate enables comprehensive compliance management and visibility across your infrastructure, making Chef ideal for enterprise-scale automation.