Grafana Loki is a log aggregation system that indexes only a small set of labels per log stream instead of the full text, which makes it cheap to run and simple to operate. For years Promtail was the standard agent for shipping logs to Loki; Grafana deprecated it in 2025 in favor of Grafana Alloy, and Promtail no longer receives updates. In this tutorial you will install Loki on Ubuntu 24.04, collect systemd journal and file logs with Alloy, convert an existing Promtail configuration to Alloy, and explore your logs with LogQL in Grafana.

Prerequisites

To follow this guide you need:

  • A server running Ubuntu 24.04 LTS with at least 2 GB of RAM, for example a CubePath VPS.
  • A non-root user with sudo privileges.
  • UFW enabled, with SSH allowed.

In this guide Loki, Alloy and Grafana all run on the same server. To collect logs from more servers, install only Alloy on each of them and point it at the Loki server, as described in Step 3.

Step 1 - Adding the Grafana package repository

Loki, Alloy, Promtail and Grafana are all published in Grafana's APT repository. Download its signing key into the keyrings directory:

sudo apt update
sudo apt install -y gpg wget jq
sudo install -m 0755 -d /etc/apt/keyrings
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null

Add the repository and refresh the package index:

echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update

Confirm that the packages are available:

apt-cache policy loki alloy grafana | grep -E '^[a-z]|Candidate'
loki:
  Candidate: 3.5.5
alloy:
  Candidate: 1.10.2
grafana:
  Candidate: 12.2.0

Your version numbers will be newer or different.

Step 2 - Installing and configuring Loki

Install Loki:

sudo apt install -y loki

The package creates a loki system user and a service that reads /etc/loki/config.yml. The default file stores data under /tmp, which is cleared on reboot, so replace it with a configuration that keeps data in /var/lib/loki. Create the directory first:

sudo mkdir -p /var/lib/loki
sudo chown loki:loki /var/lib/loki

Open the configuration file:

sudo nano /etc/loki/config.yml

Replace its content with:

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  instance_addr: 127.0.0.1
  path_prefix: /var/lib/loki
  storage:
    filesystem:
      chunks_directory: /var/lib/loki/chunks
      rules_directory: /var/lib/loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2024-01-01
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  retention_period: 744h

compactor:
  working_directory: /var/lib/loki/compactor
  retention_enabled: true
  delete_request_store: filesystem

This runs Loki as a single process (monolithic mode) with the local filesystem as storage:

  • auth_enabled: false disables multi-tenancy, so clients do not need to send a tenant ID.
  • schema_config uses the TSDB index with schema v13, the current recommended format. The from date must be in the past; never change an existing entry, only add new ones.
  • retention_period: 744h keeps logs for 31 days, and the compactor deletes older data because retention_enabled is set.

Restart Loki and enable it at boot:

sudo systemctl restart loki
sudo systemctl enable loki

Loki takes about 15 seconds to become ready. Check its readiness endpoint:

curl -s http://localhost:3100/ready
ready

If it answers Ingester not ready, wait a few seconds and try again. If the service failed, read sudo journalctl -u loki -n 50; YAML indentation mistakes are the most common cause.

Step 3 - Collecting logs with Grafana Alloy

Alloy is Grafana's collector for logs, metrics and traces. Its configuration is a set of connected components: some find or read logs, others process them, and a writer sends them to Loki. Install it:

sudo apt install -y alloy

Alloy runs as the alloy user. Add it to the adm group to read files in /var/log, and to systemd-journal to read the journal:

sudo usermod -aG adm,systemd-journal alloy

Open the Alloy configuration file:

sudo nano /etc/alloy/config.alloy

Replace its content with:

// Send everything to the local Loki
loki.write "local" {
  endpoint {
    url = "http://localhost:3100/loki/api/v1/push"
  }
}

// Systemd journal: one stream per unit
loki.relabel "journal" {
  forward_to = []

  rule {
    source_labels = ["__journal__systemd_unit"]
    target_label  = "unit"
  }
}

loki.source.journal "journal" {
  forward_to    = [loki.write.local.receiver]
  relabel_rules = loki.relabel.journal.rules
  labels        = {job = "systemd-journal", host = constants.hostname}
}

// Plain log files
local.file_match "files" {
  path_targets = [
    {"__path__" = "/var/log/syslog", "job" = "syslog", "host" = constants.hostname},
    {"__path__" = "/var/log/auth.log", "job" = "auth", "host" = constants.hostname},
  ]
}

loki.source.file "files" {
  targets    = local.file_match.files.targets
  forward_to = [loki.write.local.receiver]
}

How this works:

  • loki.source.journal reads the systemd journal. Journal fields are exposed as hidden labels starting with __journal_, and loki.relabel copies the systemd unit name into a unit label.
  • local.file_match lists the files to tail and their labels, and loki.source.file reads them. Alloy remembers the read position of each file across restarts.
  • Every source forwards to loki.write.local.receiver, which pushes to Loki's API.

Keep labels to a few low-cardinality values such as job, host and unit. Values that change per line, such as user IDs or request paths, belong in the log line and are parsed at query time (Step 5); turning them into labels creates huge numbers of streams and slows Loki down.

Check the syntax, then restart Alloy:

sudo alloy fmt /etc/alloy/config.alloy > /dev/null && echo "syntax OK"
sudo systemctl restart alloy
sudo systemctl status alloy --no-pager

Confirm that Loki is receiving streams by asking for the label names it knows:

curl -s http://localhost:3100/loki/api/v1/labels | jq
{
  "status": "success",
  "data": [
    "filename",
    "host",
    "job",
    "service_name",
    "unit"
  ]
}

Alloy also has a debugging UI on 127.0.0.1:12345 that shows each component and its health. To view it from your workstation, open an SSH tunnel with ssh -L 12345:localhost:12345 your_user@your_server_ip and browse to http://localhost:12345.

To ship logs from other servers, install Alloy on each one with Steps 1 and 3, and change the url in loki.write to http://your_loki_server_ip:3100/loki/api/v1/push.

Step 4 - Migrating an existing Promtail configuration

If you already run Promtail, Alloy can convert its configuration automatically. On the server running Promtail, install Alloy (Steps 1 and 3) and run:

sudo alloy convert --source-format=promtail --output=/etc/alloy/config.alloy /etc/promtail/config.yml

The command translates scrape_configs, relabel_configs, pipeline_stages and clients into the equivalent Alloy components and prints a warning for anything it cannot convert. Review the result with sudo nano /etc/alloy/config.alloy, then switch agents so the files are not shipped twice:

sudo systemctl disable --now promtail
sudo systemctl restart alloy

Alloy keeps its own read positions and does not reuse Promtail's positions.yaml, so it may send some recent lines again once. After confirming that logs are arriving, remove Promtail with sudo apt remove promtail.

Step 5 - Querying logs with LogQL

LogQL is Loki's query language. A query always starts with a stream selector in braces that picks streams by label, followed by optional filters and parsers. You can run queries from the shell against the API. Show the five most recent lines from the SSH service:

curl -G -s http://localhost:3100/loki/api/v1/query_range \
  --data-urlencode 'query={job="systemd-journal", unit="ssh.service"}' \
  --data-urlencode 'limit=5' | jq -r '.data.result[].values[][1]'
Accepted publickey for your_user from 203.0.113.10 port 51514 ssh2: ED25519 SHA256:...
pam_unix(sshd:session): session opened for user your_user(uid=1000) by your_user(uid=0)
...

Grafana, set up in the next step, is the usual place to write queries. Useful LogQL patterns:

QueryWhat it does
{job="auth"} |= "Failed password"Lines containing a string (!= excludes, |~ uses a regex)
{job="auth"} |= "Failed password" | pattern "<_> from <ip> port <_>"Extracts ip from each line at query time
{job="systemd-journal"} | json | level="error"Parses JSON log lines and filters on a field
sum by (unit) (count_over_time({job="systemd-journal"}[5m]))Log lines per unit over 5-minute windows
topk(5, sum by (ip) (count_over_time({job="auth"} |= "Failed password" | pattern "<_> from <ip> port <_>" [1h])))Top 5 IPs with failed SSH logins in the last hour

The last two are metric queries: they turn log lines into numbers you can graph or alert on.

Step 6 - Visualizing logs in Grafana

Install Grafana and start it:

sudo apt install -y grafana
sudo systemctl enable --now grafana-server

Instead of adding the Loki data source by hand, provision it from a file so it survives reinstalls. Create the file:

sudo nano /etc/grafana/provisioning/datasources/loki.yaml
apiVersion: 1

datasources:
  - name: Loki
    type: loki
    access: proxy
    url: http://localhost:3100
    isDefault: true

Restart Grafana to load it, and allow access to port 3000 from your IP only, replacing your_ip:

sudo systemctl restart grafana-server
sudo ufw allow from your_ip to any port 3000 proto tcp

Open http://your_server_ip:3000, log in as admin with the password admin, and set a new password when prompted. Then:

  1. Open Explore, which already has the Loki data source selected.
  2. Switch the query editor to Code and enter {job="auth"}, then click Run query. You see the log lines and a histogram of their volume.
  3. Try the metric query sum by (unit) (count_over_time({job="systemd-journal"}[5m])) and switch the visualization to a time series.

To keep a view, click Add to dashboard from Explore. A dashboard with a Logs panel for {job="auth"} next to a time series of failed logins gives a quick security overview of the server.

Troubleshooting

Loki fails to start with mkdir ... permission denied: the loki user cannot write to the paths in the configuration. Check that /var/lib/loki exists and is owned by loki:loki.

Alloy runs but no logs arrive: open the Alloy UI or run sudo journalctl -u alloy -n 50. permission denied on /var/log files or the journal means the group change in Step 3 has not taken effect; restart Alloy after usermod.

Grafana shows No data: widen the time range, check that the labels in your selector exist with curl -s localhost:3100/loki/api/v1/labels, and confirm the data source test succeeds under Connections > Data sources > Loki.

Loki returns too many outstanding requests or queries are slow: the stream selector is too broad or there are too many streams. Narrow the selector with more labels and check that you did not turn a high-cardinality value into a label.

Conclusion

Loki now stores logs from the systemd journal and your log files, Alloy ships them in place of the deprecated Promtail, and Grafana lets you search them and turn them into graphs with LogQL. As next steps, install Alloy on your other servers, add Grafana alert rules on metric queries such as failed login counts, and move Loki's storage to S3-compatible object storage when your log volume outgrows a single disk.