RabbitMQ Installation and Configuration

RabbitMQ is a robust, feature-rich message broker that implements the Advanced Message Queuing Protocol (AMQP). It provides reliable message delivery, flexible routing, clustering capabilities, and a comprehensive management interface. This guide covers installation, basic configuration, user management, and setting up exchanges, queues, and virtual hosts on Linux systems.

Table of Contents

• Prerequisites
• Installing Erlang/OTP
• Installing RabbitMQ
• Basic Configuration
• Managing Users and Virtual Hosts
• Setting Up Exchanges and Queues
• RabbitMQ Management UI
• Policies and Rate Limiting
• Testing Message Flow
• Troubleshooting
• Conclusion

Prerequisites

Before installing RabbitMQ, ensure you have:

• A Linux system (Ubuntu 20.04+ or CentOS 8+)
• Root or sudo access
• At least 2GB RAM available
• Internet connectivity for package downloads
• Basic knowledge of message queues and AMQP concepts

Installing Erlang/OTP

RabbitMQ runs on the Erlang runtime, so you must install it first. Erlang/OTP provides the foundation for RabbitMQ's concurrency model.

For Ubuntu/Debian systems, add the RabbitMQ repository which includes Erlang packages:

curl -1sLf 'https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA' | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null
sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
deb [signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://ppa.launchpadcontent.net/rabbitmq/rabbitmq-erlang/ubuntu focal main
deb-src [signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://ppa.launchpadcontent.net/rabbitmq/rabbitmq-erlang/ubuntu focal main
EOF
sudo apt-get update

Install the Erlang package:

sudo apt-get install -y erlang-nox

Verify the installation:

erl -version

For CentOS/RHEL, enable the EPEL repository and install Erlang:

sudo yum install -y centos-release-scl
sudo yum install -y erlang
erl -version

Installing RabbitMQ

After Erlang is installed, proceed with RabbitMQ installation. Add the official RabbitMQ repository:

sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
deb [signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://dl.bintray.com/rabbitmq-erlang/debian focal erlang
deb [signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://dl.bintray.com/rabbitmq/debian focal main
EOF

Update the package cache and install RabbitMQ:

sudo apt-get update
sudo apt-get install -y rabbitmq-server

Start the RabbitMQ service and enable it to start on boot:

sudo systemctl start rabbitmq-server
sudo systemctl enable rabbitmq-server

Verify the service is running:

sudo systemctl status rabbitmq-server
sudo rabbitmqctl version

The output should display the RabbitMQ and Erlang versions installed.

Basic Configuration

The main RabbitMQ configuration file is located at /etc/rabbitmq/rabbitmq.conf. Create or modify this file to customize RabbitMQ behavior:

sudo nano /etc/rabbitmq/rabbitmq.conf

Here are essential configuration options:

# Network and protocol settings
listeners.tcp.default = 5672

# Management plugin port
management.tcp.port = 15672

# Memory threshold - trigger memory alarm at 80% RAM usage
vm_memory_high_watermark.relative = 0.8

# Disk space threshold
disk_free_limit.absolute = 2GB

# Default vhost and user (created automatically)
default_vhost = /
default_user = guest
default_pass = guest

# Maximum connections per channel
channel_max = 2048

# Maximum channels per connection
max_channels = 4096

# Heartbeat timeout in seconds
heartbeat = 60

# Log file location
log.file.level = info
log.console.level = info

# Mnesia database directory
mnesia.dir = /var/lib/rabbitmq/mnesia

After modifying the configuration, restart RabbitMQ:

sudo systemctl restart rabbitmq-server

Managing Users and Virtual Hosts

RabbitMQ uses virtual hosts (vhosts) to isolate message brokers and users to control access. By default, a guest user exists but should be removed in production.

List all users:

sudo rabbitmqctl list_users

Create a new administrative user:

sudo rabbitmqctl add_user admin_user secure_password123
sudo rabbitmqctl set_user_tags admin_user administrator

Create a new application user with limited permissions:

sudo rabbitmqctl add_user app_user app_password456
sudo rabbitmqctl set_user_tags app_user none

Create a virtual host for application isolation:

sudo rabbitmqctl add_vhost production_app

Grant permissions to users on specific vhosts. The permission format is configure/write/read:

# Grant admin user full access to production_app vhost
sudo rabbitmqctl set_permissions -p production_app admin_user ".*" ".*" ".*"

# Grant app_user limited access - can only write and read
sudo rabbitmqctl set_permissions -p production_app app_user "" ".*" ".*"

View user permissions:

sudo rabbitmqctl list_user_permissions admin_user
sudo rabbitmqctl list_permissions -p production_app

Delete the default guest user in production:

sudo rabbitmqctl delete_user guest

Change a user's password:

sudo rabbitmqctl change_password admin_user new_secure_password

Setting Up Exchanges and Queues

Exchanges are message routing entities, and queues are message storage. Use the RabbitMQ management UI or command-line tools to create them.

Using the management UI (covered in the next section) provides a graphical interface. Alternatively, use AMQP clients or the HTTP API.

Here's an example using Python with pika library to create exchanges and queues:

sudo apt-get install -y python3-pip
pip3 install pika

Create a Python script to set up messaging infrastructure:

#!/usr/bin/env python3
import pika
import sys

# Connect to RabbitMQ
credentials = pika.PlainCredentials('app_user', 'app_password456')
connection = pika.BlockingConnection(pika.ConnectionParameters(
    host='localhost',
    port=5672,
    virtual_host='production_app',
    credentials=credentials
))
channel = connection.channel()

# Declare a durable topic exchange
channel.exchange_declare(
    exchange='orders',
    exchange_type='topic',
    durable=True,
    auto_delete=False
)

# Declare durable queues
channel.queue_declare(queue='order_processing', durable=True)
channel.queue_declare(queue='order_notifications', durable=True)

# Bind queues to exchange with routing keys
channel.queue_bind(
    exchange='orders',
    queue='order_processing',
    routing_key='order.create'
)

channel.queue_bind(
    exchange='orders',
    queue='order_notifications',
    routing_key='order.*'
)

print("Exchanges and queues configured successfully")
connection.close()

Run the script:

python3 setup_rabbitmq.py

RabbitMQ Management UI

RabbitMQ includes a management plugin providing a web-based interface for administration. Enable it:

sudo rabbitmq-plugins enable rabbitmq_management

The management UI is accessible at http://localhost:15672 or http://your-server-ip:15672. Log in with your admin credentials.

From the management UI, you can:

• Create and delete exchanges, queues, and bindings
• Manage users and permissions
• Monitor message flow and queue depth
• Set up policies for high availability
• View connection and channel statistics

Navigate the tabs:

• Overview: Cluster status and message rates
• Connections: Active AMQP connections
• Channels: Active channels with memory usage
• Queues: Queue details, message counts, consumers
• Admin: User and vhost management

To reset the management database (use cautiously):

sudo rabbitmqctl reset
sudo rabbitmqctl start_app

Policies and Rate Limiting

Policies allow you to apply configurations to queues and exchanges matching certain criteria. They're essential for high-availability setups and rate limiting.

Set a policy to enable queue mirroring across cluster nodes:

sudo rabbitmqctl set_policy -p production_app ha-all "^" \
  '{"ha-mode":"all","ha-sync-batch-size":5}' \
  --priority 0 \
  --apply-to queues

This policy applies to all queues matching the pattern ^ (all queues) in the production_app vhost.

Set rate limiting policy on queues:

sudo rabbitmqctl set_policy -p production_app rate-limit "^limited.*" \
  '{"max-length":10000,"message-ttl":300000}' \
  --priority 10 \
  --apply-to queues

List all policies:

sudo rabbitmqctl list_policies -p production_app

Clear a policy:

sudo rabbitmqctl clear_policy -p production_app ha-all

Testing Message Flow

Verify your RabbitMQ setup by publishing and consuming test messages. Create a test publisher script:

#!/usr/bin/env python3
import pika
import json
from datetime import datetime

credentials = pika.PlainCredentials('app_user', 'app_password456')
connection = pika.BlockingConnection(pika.ConnectionParameters(
    host='localhost',
    virtual_host='production_app',
    credentials=credentials
))
channel = connection.channel()

message = {
    'order_id': '12345',
    'customer': 'John Doe',
    'timestamp': datetime.now().isoformat(),
    'total': 99.99
}

channel.basic_publish(
    exchange='orders',
    routing_key='order.create',
    body=json.dumps(message),
    properties=pika.BasicProperties(
        delivery_mode=2,  # Make message persistent
        content_type='application/json'
    )
)

print(f"Message sent: {json.dumps(message, indent=2)}")
connection.close()

Create a consumer script:

#!/usr/bin/env python3
import pika
import json

def callback(ch, method, properties, body):
    message = json.loads(body)
    print(f"Received: {json.dumps(message, indent=2)}")
    ch.basic_ack(delivery_tag=method.delivery_tag)

credentials = pika.PlainCredentials('app_user', 'app_password456')
connection = pika.BlockingConnection(pika.ConnectionParameters(
    host='localhost',
    virtual_host='production_app',
    credentials=credentials
))
channel = connection.channel()

channel.basic_consume(
    queue='order_processing',
    on_message_callback=callback
)

print("Waiting for messages...")
channel.start_consuming()

Test the flow:

# Terminal 1
python3 consumer.py

# Terminal 2
python3 publisher.py

Troubleshooting

Check RabbitMQ logs for errors:

sudo tail -f /var/log/rabbitmq/[email protected]

Verify RabbitMQ is listening on expected ports:

sudo netstat -tuln | grep -E '5672|15672'

Check cluster node status:

sudo rabbitmqctl cluster_status

Monitor memory and disk usage:

sudo rabbitmqctl status | grep -A 10 "memory\|disk"

Reset RabbitMQ completely (destructive):

sudo systemctl stop rabbitmq-server
sudo rm -rf /var/lib/rabbitmq/mnesia/*
sudo systemctl start rabbitmq-server

Conclusion

RabbitMQ provides a powerful, reliable message brokering platform suitable for distributed systems, microservices, and event-driven architectures. This guide covered installation, configuration, user and vhost management, exchange and queue setup, and basic testing. For production deployments, implement clustering for high availability, configure persistent storage, implement monitoring with Prometheus, and set up backup procedures. Regular monitoring of queue depths, memory usage, and connection counts will help maintain optimal performance in your messaging infrastructure.