Microservices Communication: REST, RabbitMQ, Redis Streams

Once an application is split into services, the way those services talk to each other decides how it behaves under load and during failures. A service that calls three others synchronously is only as available as the weakest of them, while a service that publishes an event and moves on keeps working when consumers are down, at the cost of eventual consistency. This guide compares the main patterns, shows each one working on Ubuntu 24.04 (an Nginx gateway for synchronous calls, RabbitMQ for work queues, Redis Streams for event logs) and covers the saga and outbox patterns for workflows that span several services.

Prerequisites

To try the examples you will need:

• A server running Ubuntu 24.04 LTS, for example a CubePath VPS, with at least 2 GB of RAM.
• A non-root user with sudo privileges.
• Basic familiarity with HTTP APIs. No application code is required: every example is driven from the command line.

Synchronous and asynchronous communication at a glance

A useful default: use synchronous calls for queries and for the part of a request the user is waiting on, and publish events for everything that can happen afterwards.

Pattern 1: Synchronous calls through an API gateway

In the synchronous pattern a client, or another service, sends an HTTP or gRPC request and blocks until it gets an answer. Placing a reverse proxy in front of the services gives clients one entry point and centralizes timeouts, retries and request IDs. Install Nginx:

sudo apt update
sudo apt install -y nginx

Create a gateway configuration that routes by path to two services, each running two instances:

sudo nano /etc/nginx/conf.d/gateway.conf

upstream users_svc {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=15s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=15s;
    keepalive 16;
}

upstream orders_svc {
    least_conn;
    server 10.0.0.21:8080 max_fails=3 fail_timeout=15s;
    server 10.0.0.22:8080 max_fails=3 fail_timeout=15s;
    keepalive 16;
}

server {
    listen 80;
    server_name api.your_domain;

    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Request-ID $request_id;
    add_header X-Request-ID $request_id always;

    proxy_connect_timeout 2s;
    proxy_read_timeout 10s;
    proxy_next_upstream error timeout http_502 http_503;
    proxy_next_upstream_tries 2;

    location /api/users/ {
        proxy_pass http://users_svc;
    }

    location /api/orders/ {
        proxy_pass http://orders_svc;
    }
}

Replace the 10.0.0.x addresses with the private IPs and ports of your services. The settings that matter for service-to-service traffic:

• Short timeouts. A 2 second connect timeout and a 10 second read timeout stop a slow service from holding every gateway connection. Set them from the real latency of each service, not from defaults.
• Retries only where they are safe. proxy_next_upstream sends a failed request to the other instance. Nginx never retries POST, PATCH or LOCK requests by default, because repeating them could, for example, create an order twice. Keep it that way unless the endpoint is idempotent.
• Passive failure detection. After three failures within 15 seconds an instance is taken out of rotation for 15 seconds. This works like a basic circuit breaker. Full circuit breakers with half-open states live in client libraries or a service mesh.
• Request IDs. $request_id is a random ID generated per request. Services should log it and forward it on their own outgoing calls so you can follow one request across every log.

Test the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Any response from the gateway now carries the ID you can search for in the service logs:

curl -si -H "Host: api.your_domain" http://127.0.0.1/api/users/ | grep -i x-request-id

X-Request-ID: 3f1c2a9d8e7b46a0b5c4d3e2f1a0b9c8

gRPC follows the same model with typed contracts and HTTP/2. Nginx proxies it with grpc_pass instead of proxy_pass.

Pattern 2: Work queues with RabbitMQ

With a message broker, the producer hands a message to the broker and returns immediately. The broker stores it until a consumer takes it and acknowledges it. If the consumer is down, messages accumulate and are processed when it comes back. RabbitMQ routes messages through exchanges to queues using a routing key. A topic exchange matches patterns such as order.*.

Install RabbitMQ from the Ubuntu repositories and enable the management plugin, which adds a web UI and an HTTP API on port 15672:

sudo apt install -y rabbitmq-server
sudo rabbitmq-plugins enable rabbitmq_management

Create a virtual host for the application and a user that can only access it. Replace your_strong_password with a real password:

sudo rabbitmqctl add_vhost orders
sudo rabbitmqctl add_user orders_app 'your_strong_password'
sudo rabbitmqctl set_permissions -p orders orders_app ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags orders_app management

The management tag allows the user to use the HTTP API, which the next commands rely on. Declare a durable topic exchange, a durable queue for the notification service, and a binding that sends every order.* message to that queue:

curl -s -u orders_app:your_strong_password -X PUT -H "content-type: application/json" \
  http://127.0.0.1:15672/api/exchanges/orders/order-events -d '{"type": "topic", "durable": true}'

curl -s -u orders_app:your_strong_password -X PUT -H "content-type: application/json" \
  http://127.0.0.1:15672/api/queues/orders/notifications -d '{"durable": true}'

curl -s -u orders_app:your_strong_password -X POST -H "content-type: application/json" \
  http://127.0.0.1:15672/api/bindings/orders/e/order-events/q/notifications -d '{"routing_key": "order.*"}'

Publish an order.created event. delivery_mode: 2 marks the message as persistent so it survives a broker restart:

curl -s -u orders_app:your_strong_password -X POST -H "content-type: application/json" \
  http://127.0.0.1:15672/api/exchanges/orders/order-events/publish -d '{
    "properties": {"delivery_mode": 2, "content_type": "application/json"},
    "routing_key": "order.created",
    "payload": "{\"order_id\": \"123\", \"total\": 99.90}",
    "payload_encoding": "string"
  }'

{"routed":true}

routed: true means at least one queue received the message. No consumer is running, so it waits in the queue:

sudo rabbitmqctl list_queues -p orders name messages consumers

Timeout: 60.0 seconds ...
Listing queues for vhost orders ...
name	messages	consumers
notifications	1	0

Take the message out of the queue as a consumer would:

curl -s -u orders_app:your_strong_password -X POST -H "content-type: application/json" \
  http://127.0.0.1:15672/api/queues/orders/notifications/get \
  -d '{"count": 1, "ackmode": "ack_requeue_false", "encoding": "auto"}'

The response contains the payload you published, and list_queues now shows 0 messages. The HTTP publish and get endpoints are meant for testing. Real services use an AMQP client library (for example pika for Python or amqplib for Node.js) and should:

• Acknowledge a message only after it has been fully processed, so a crash causes redelivery instead of loss.
• Use publisher confirms so the producer knows the broker stored the message.
• Set a prefetch limit (basic.qos) so one slow consumer does not hoard messages.
• Send messages that keep failing to a dead letter exchange instead of retrying them forever.

The management port should not be public. UFW blocks it unless you open it; to reach the UI, use an SSH tunnel such as ssh -L 15672:127.0.0.1:15672 your_user@your_server_ip and browse to http://localhost:15672.

Pattern 3: Event logs with Redis Streams

A queue delivers each message to one consumer and then forgets it. An event log keeps events in order and lets any number of consumer groups read the same stream independently, each with its own position. That fits events such as order.created that billing, notifications and analytics all need. Redis Streams provide this with a small footprint; Kafka does the same at much larger scale.

Install Redis:

sudo apt install -y redis-server

Create two consumer groups on the order-events stream. $ means "only events added from now on" and MKSTREAM creates the stream if it does not exist:

redis-cli XGROUP CREATE order-events billing '$' MKSTREAM
redis-cli XGROUP CREATE order-events notifications '$' MKSTREAM

OK
OK

Append an event. * lets Redis assign an ID based on the current time:

redis-cli XADD order-events '*' type order.created order_id 123 total 99.90

"1727254800123-0"

Read it as worker worker-1 of the billing group. The > ID means "messages never delivered to this group":

redis-cli XREADGROUP GROUP billing worker-1 COUNT 10 STREAMS order-events '>'

1) 1) "order-events"
   2) 1) 1) "1727254800123-0"
         2) 1) "type"
            2) "order.created"
            3) "order_id"
            4) "123"
            5) "total"
            6) "99.90"

Running the same command with GROUP notifications worker-1 returns the same event, because each group tracks its own position. Within a group, adding worker-2 splits the events between the two workers.

Until a worker acknowledges an event, it stays in the group's pending list. That is how Redis Streams avoid losing events when a worker crashes. Check the pending count, then acknowledge with the ID from the previous output:

redis-cli XPENDING order-events billing
redis-cli XACK order-events billing 1727254800123-0

1) (integer) 1
2) "1727254800123-0"
3) "1727254800123-0"
4) 1) 1) "worker-1"
      2) "1"
(integer) 1

In production, a worker that restarts should first reclaim events left pending by dead workers with XAUTOCLAIM, and the producer should cap the stream's length (for example XADD order-events MAXLEN '~' 1000000 '*' ...) so memory does not grow without limit.

Pattern 4: Sagas for workflows across services

Placing an order might reserve stock in the inventory service, charge the card in the payment service and schedule shipping. Each service owns its own database, so there is no single transaction that covers all three. A saga splits the workflow into local transactions, each with a compensating action that undoes it if a later step fails:

If the payment fails in step 3, the saga runs the compensations for steps 2 and 1 in reverse order: release the stock, then cancel the order.

There are two ways to coordinate a saga:

• Choreography. Each service listens for events and publishes its own: inventory reacts to order.created and publishes inventory.reserved, payments reacts to that, and so on. There is no central component, but the full workflow is only visible by reading every service, so it suits short sagas of two or three steps.
• Orchestration. One component (often part of the orders service) holds the state of each saga, sends commands to each service and decides which compensations to run. It is easier to follow, monitor and change, and is the better choice as the number of steps grows.

Two supporting patterns make sagas reliable:

• Transactional outbox. A service must not update its database and then publish an event as two separate operations, because a crash between them loses the event. Instead, write the event to an outbox table in the same database transaction as the business change, and let a separate process publish rows from that table to the broker.
• Idempotent consumers. Brokers deliver at least once, so a consumer can see the same event twice. Record the ID of every processed event and skip duplicates. With Redis, SET ... NX does this in one atomic call:

redis-cli SET processed:billing:1727254800123-0 1 NX EX 86400
redis-cli SET processed:billing:1727254800123-0 1 NX EX 86400

OK
(nil)

The first call returns OK, so the consumer processes the event. The second returns (nil), so the event is a duplicate and the consumer only acknowledges it.

Finding services

The gateway example uses fixed IP addresses. Once instances come and go, services need discovery. Docker Compose and Kubernetes provide it through built-in DNS (http://users:8080 resolves to the current containers or pods), which covers most deployments. Outside those platforms, a registry such as Consul keeps a list of healthy instances that clients or the gateway query.

Which pattern should you choose?

Start with synchronous calls and a gateway, which are the easiest to build and debug, and introduce a queue or an event log when a concrete need appears: a slow task blocking responses, a service that must keep working while another is down, or a third consumer interested in the same event.

Troubleshooting

Nginx returns 502 or 504 for one service: the gateway cannot reach its instances or they answer too slowly. The error log names the upstream (sudo tail -n 50 /var/log/nginx/error.log). Test an instance directly with curl http://10.0.0.11:8080/ from the gateway.

Messages pile up in a RabbitMQ queue: check that consumers are connected and whether messages are stuck unacknowledged, which usually means a consumer that crashes before calling ack:

sudo rabbitmqctl list_queues -p orders name messages_ready messages_unacknowledged consumers

The HTTP API returns {"error":"not_authorised"}: the user is missing the management tag or permissions on the vhost. Rerun the set_user_tags and set_permissions commands.

Redis Streams pending list keeps growing: workers are reading but not calling XACK, or a worker died. Inspect the pending entries per consumer with redis-cli XPENDING order-events billing - + 10 and reclaim old ones with XAUTOCLAIM.

Conclusion

You compared synchronous and asynchronous communication, routed synchronous calls through an Nginx gateway with sane timeouts and retries, moved background work onto a RabbitMQ queue, fanned events out to several consumer groups with Redis Streams, and saw how sagas, the outbox pattern and idempotent consumers keep multi-service workflows consistent. Next, pick one real workflow in your application and move its non-blocking part to a queue, add request ID logging to every service, and set up alerts on queue depth and pending stream entries.