ZeroMQ on Ubuntu 24.04: Messaging Patterns with Python

ZeroMQ (also written ØMQ or ZMQ) is a messaging library, not a server: it gives your programs sockets that carry whole messages and implement patterns such as request-reply, publish-subscribe and work pipelines, with no broker in the middle. In this tutorial you will install ZeroMQ and its Python bindings on Ubuntu 24.04, write a small working program for each of the three core patterns, and then run a publisher and subscriber on two different servers.

Prerequisites

To follow this guide you need:

• A server running Ubuntu 24.04 LTS, for example a CubePath VPS, and a non-root user with sudo privileges.
• Basic knowledge of Python 3.
• For Step 6 only: a second Ubuntu 24.04 server that can reach the first one over the network.

Most examples run on one machine. Open two or three SSH sessions to the same server, one per program.

Step 1 - Installing ZeroMQ and pyzmq

Ubuntu packages both the C library (libzmq5) and the Python bindings (python3-zmq). Installing the bindings pulls in the library as a dependency:

sudo apt update
sudo apt install python3-zmq

If you also plan to compile C or C++ programs against ZeroMQ, install the development headers as well:

sudo apt install libzmq3-dev

Verify that Python can load the library and print both versions:

python3 -c 'import zmq; print("libzmq", zmq.zmq_version(), "pyzmq", zmq.pyzmq_version())'

libzmq 4.3.5 pyzmq 24.0.1

Create a working directory for the examples:

mkdir ~/zmq-demo && cd ~/zmq-demo

Step 2 - Understanding sockets and patterns

A few ideas make the rest of the examples easy to follow:

• Context: one zmq.Context() per process. It owns the background I/O threads; all sockets are created from it.
• Socket type: each socket has a type that decides its behavior. Types only work in matching pairs.
• bind and connect: the stable side of a connection calls bind() on an endpoint, the other side calls connect(). Unlike TCP, either side can be started first; ZeroMQ reconnects automatically.
• Transports: tcp:// between hosts, ipc:// between processes on the same host, inproc:// between threads in one process.
• Messages: ZeroMQ delivers whole messages, never partial ones. A message can have several frames (send_multipart).

DEALER and ROUTER are asynchronous versions of REQ and REP used to build brokers and load balancers; they are beyond the scope of this introduction.

Step 3 - Building a request-reply service

A REP socket receives a request and must send exactly one reply before it can receive the next. A REQ socket does the opposite. Create the server:

nano rep_server.py

import zmq

context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:5555")
print("REP server listening on port 5555")

try:
    while True:
        request = socket.recv_string()
        print(f"received: {request}")
        socket.send_string(request.upper())
except KeyboardInterrupt:
    pass
finally:
    socket.close()
    context.term()

Now the client. If the server is down, a plain recv() would block forever, so the client sets a receive timeout. After a timeout a REQ socket is stuck waiting for a reply, which is why the client closes it and gives up instead of sending again on the same socket:

nano req_client.py

import zmq

context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.setsockopt(zmq.RCVTIMEO, 3000)  # wait at most 3 seconds for a reply
socket.setsockopt(zmq.LINGER, 0)       # do not block on close if the server is gone
socket.connect("tcp://localhost:5555")

try:
    for word in ["hello", "zeromq", "patterns"]:
        socket.send_string(word)
        try:
            reply = socket.recv_string()
        except zmq.Again:
            print("no reply from server, giving up")
            break
        print(f"{word} -> {reply}")
finally:
    socket.close()
    context.term()

Start the server in the first session:

python3 rep_server.py

Run the client in a second session:

cd ~/zmq-demo
python3 req_client.py

hello -> HELLO
zeromq -> ZEROMQ
patterns -> PATTERNS

Stop the server with Ctrl+C and run the client again. After three seconds it prints no reply from server, giving up instead of hanging.

Step 4 - Publishing and subscribing to topics

A PUB socket sends every message to all connected subscribers. A SUB socket must subscribe to at least one prefix, otherwise it receives nothing. Filtering is done on the beginning of the first frame, so a common convention is to send the topic as the first frame and the payload as the second.

Create the publisher:

nano publisher.py

import json
import random
import time

import zmq

context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:5556")
print("publisher bound to port 5556")

try:
    while True:
        topic = random.choice(["weather", "sports", "news"])
        payload = json.dumps({"value": random.randint(10, 30), "ts": time.time()})
        socket.send_multipart([topic.encode(), payload.encode()])
        time.sleep(1)
except KeyboardInterrupt:
    pass
finally:
    socket.close()
    context.term()

Create a subscriber that listens only to weather and news:

nano subscriber.py

import sys

import zmq

endpoint = sys.argv[1] if len(sys.argv) > 1 else "tcp://localhost:5556"

context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect(endpoint)
socket.setsockopt(zmq.SUBSCRIBE, b"weather")
socket.setsockopt(zmq.SUBSCRIBE, b"news")
print(f"subscribed to weather and news on {endpoint}")

try:
    while True:
        topic, payload = socket.recv_multipart()
        print(f"[{topic.decode()}] {payload.decode()}")
except KeyboardInterrupt:
    pass
finally:
    socket.close()
    context.term()

Start the publisher in one session and the subscriber in another:

python3 publisher.py

cd ~/zmq-demo
python3 subscriber.py

The subscriber only prints the topics it asked for; sports messages are filtered out:

subscribed to weather and news on tcp://localhost:5556
[news] {"value": 17, "ts": 1790000000.12}
[weather] {"value": 24, "ts": 1790000002.13}

Publish-subscribe in ZeroMQ is fire-and-forget. Messages sent before a subscriber has finished connecting (the "slow joiner" effect) and messages that arrive while a subscriber is down are lost. When every message matters, use a pipeline or request-reply, or a persistent broker such as RabbitMQ or Kafka.

Step 5 - Distributing work with a push-pull pipeline

A pipeline has three parts: a ventilator that PUSHes jobs, workers that PULL jobs and PUSH results, and a sink that PULLs the results. ZeroMQ spreads jobs round-robin across the connected workers.

Create the ventilator. It waits for you to press Enter so all workers have time to connect; otherwise the first worker to connect would receive the whole batch:

nano ventilator.py

import zmq

context = zmq.Context()
sender = context.socket(zmq.PUSH)
sender.bind("tcp://*:5557")

input("Start the workers and the sink, then press Enter to send 20 jobs...")
for job_id in range(1, 21):
    sender.send_json({"job_id": job_id, "n": job_id * 1000})
print("20 jobs sent")

sender.close()
context.term()

Create the worker. It sums the numbers from 0 to n and reports the result:

nano worker.py

import os

import zmq

context = zmq.Context()
receiver = context.socket(zmq.PULL)
receiver.connect("tcp://localhost:5557")
sender = context.socket(zmq.PUSH)
sender.connect("tcp://localhost:5558")

pid = os.getpid()
print(f"worker {pid} ready")
try:
    while True:
        job = receiver.recv_json()
        result = sum(range(job["n"] + 1))
        sender.send_json({"job_id": job["job_id"], "worker": pid, "result": result})
except KeyboardInterrupt:
    pass
finally:
    receiver.close()
    sender.close()
    context.term()

Create the sink, which collects the 20 results and shows how many each worker handled:

nano sink.py

from collections import Counter

import zmq

context = zmq.Context()
receiver = context.socket(zmq.PULL)
receiver.bind("tcp://*:5558")

per_worker = Counter()
for _ in range(20):
    msg = receiver.recv_json()
    per_worker[msg["worker"]] += 1
print(f"received 20 results: {dict(per_worker)}")

receiver.close()
context.term()

Start the sink and two workers, each in its own session:

python3 sink.py

cd ~/zmq-demo && python3 worker.py

cd ~/zmq-demo && python3 worker.py

Finally run the ventilator in a fourth session and press Enter:

cd ~/zmq-demo && python3 ventilator.py

The sink reports that the jobs were split between the two workers:

received 20 results: {41822: 10, 41830: 10}

The PIDs will differ on your system. Add more workers to spread the load further, including workers on other machines.

Step 6 - Connecting two servers

Every example above works across hosts by changing localhost to the address of the server that binds. As an example, run the publisher on the first server (your_server_ip) and the subscriber on the second.

ZeroMQ traffic is not encrypted or authenticated by default. On the publishing server, allow port 5556 only from the second server's address (your_client_ip) instead of opening it to everyone:

sudo ufw allow from your_client_ip to any port 5556 proto tcp
sudo ufw status

To                         Action      From
--                         ------      ----
5556/tcp                   ALLOW       your_client_ip

On the second server, install python3-zmq as in Step 1, copy subscriber.py to it, and point it at the first server:

python3 subscriber.py tcp://your_server_ip:5556

Messages from the publisher now appear on the second server. For traffic that crosses untrusted networks, use ZeroMQ's built-in CURVE encryption or run the connection over a private network or VPN.

Troubleshooting

• zmq.error.ZMQError: Address already in use. Another process, often a previous run of the same script, is bound to the port. Find it with sudo ss -ltnp | grep 5556 and stop it.
• The subscriber receives nothing. Check that it calls setsockopt(zmq.SUBSCRIBE, ...) with a prefix that matches the first frame, that it connects to the right host and port, and that the firewall allows the port.
• The REQ client raises Operation cannot be accomplished in current state. A REQ socket must strictly alternate send and recv. After a timeout, close the socket and create a new one before sending again.
• One worker gets all the jobs. The ventilator started sending before the other workers had connected. Start the workers first, as the ventilator's prompt suggests.

Conclusion

You installed ZeroMQ and pyzmq on Ubuntu 24.04 and built working request-reply, publish-subscribe and push-pull programs, then connected two servers with a restricted firewall rule. Next steps include the zmq.Poller class for waiting on several sockets at once, DEALER/ROUTER sockets for asynchronous brokers, and CURVE security for authenticated, encrypted links. The ZeroMQ Guide covers all of these in depth.