Vector is an open-source observability data pipeline written in Rust. It reads logs and metrics from sources, reshapes them with transforms and writes them to sinks, all from a single binary with a small memory footprint. In this tutorial you will install Vector on Ubuntu 24.04 and build a pipeline that tails the Nginx access log, parses each line into structured fields with the Vector Remap Language (VRL), drops health check noise and writes the result to daily JSON files, with an optional sink to Grafana Loki. You will also add a unit test so configuration changes can be checked before they reach production.

Prerequisites

  • A server running Ubuntu 24.04 LTS, for example a CubePath VPS. Vector itself needs very little: 512 MB of RAM is enough for this pipeline.
  • A non-root user with sudo privileges.
  • Nginx installed and serving traffic, so /var/log/nginx/access.log exists (sudo apt install nginx).
  • Optional: a Grafana Loki instance reachable from the server, if you want to ship logs to it.

How a Vector pipeline is structured

A Vector configuration is a graph of three kinds of components:

  • Sources ingest data: files, the systemd journal, syslog, Kafka, HTTP and more.
  • Transforms modify events: parse, filter, route, aggregate or turn logs into metrics.
  • Sinks send events out: files, Loki, Elasticsearch, S3, Prometheus and more.

Each component has a unique ID, and transforms and sinks list the IDs they read from in inputs. Vector buffers between components and applies back pressure when a sink is slow.

Step 1 - Installing Vector

Vector publishes APT and RPM repositories. The supported way to add them is a setup script that installs the signing keys and the repository definition. Download it and read it before running it:

curl -fsSL https://setup.vector.dev -o setup-vector.sh
less setup-vector.sh
sudo bash setup-vector.sh

Install the package:

sudo apt install -y vector

Check the installed version:

vector --version

vector 0.49.0 (x86_64-unknown-linux-gnu)

Your version will be newer or older; any recent release works with this tutorial. The package creates a vector system user, a systemd unit and a sample configuration in /etc/vector/vector.yaml.

Step 2 - Giving Vector access to the Nginx logs

On Ubuntu, Nginx logs are owned by www-data and readable by the adm group. Add the vector user to that group so the service can read them:

sudo usermod -aG adm vector

Create a directory for the JSON archive that the pipeline will write:

sudo mkdir -p /var/log/vector
sudo chown vector:vector /var/log/vector

Verify that the vector user can read the access log:

sudo -u vector head -n 1 /var/log/nginx/access.log

If this prints a log line (or nothing, on an empty log) without a Permission denied error, access is in place.

Step 3 - Writing the pipeline configuration

Keep the sample file for reference and create a new one:

sudo mv /etc/vector/vector.yaml /etc/vector/vector.yaml.sample
sudo nano /etc/vector/vector.yaml

Paste the following configuration:

data_dir: /var/lib/vector

api:
  enabled: true
  address: 127.0.0.1:8686

sources:
  nginx_access:
    type: file
    include:
      - /var/log/nginx/access.log
    read_from: end

  vector_metrics:
    type: internal_metrics

transforms:
  parse_nginx:
    type: remap
    inputs:
      - nginx_access
    drop_on_error: true
    source: |
      . = merge(., parse_nginx_log!(.message, "combined"))
      del(.message)
      .env = "production"

  drop_health:
    type: filter
    inputs:
      - parse_nginx
    condition: '.path != "/health" && .path != "/favicon.ico"'

sinks:
  archive:
    type: file
    inputs:
      - drop_health
    path: /var/log/vector/nginx-%Y-%m-%d.json
    encoding:
      codec: json

  prometheus:
    type: prometheus_exporter
    inputs:
      - vector_metrics
    address: 127.0.0.1:9598

tests:
  - name: parses a combined log line
    inputs:
      - insert_at: parse_nginx
        type: log
        log_fields:
          message: '203.0.113.10 - - [10/Mar/2026:13:55:36 +0000] "GET /pricing HTTP/1.1" 200 5120 "-" "curl/8.5.0"'
    outputs:
      - extract_from: parse_nginx
        conditions:
          - type: vrl
            source: |
              assert_eq!(.status, 200)
              assert_eq!(.path, "/pricing")
              assert_eq!(.client, "203.0.113.10")
              assert!(!exists(.message))

What each part does:

  • data_dir is where Vector keeps its checkpoints (how far it has read each file) and disk buffers.
  • api enables the local API used by vector top and vector tap. It listens on localhost only.
  • nginx_access tails the access log. read_from: end means only new lines are read the first time; after that, Vector resumes from its checkpoint and handles log rotation.
  • parse_nginx is a remap transform. parse_nginx_log! turns the raw line into fields such as client, method, path, status, size, referer and agent, and merge keeps the fields the file source added (host, file, timestamp). The ! makes the function abort on a line that does not match, and drop_on_error: true discards that event instead of passing it through unparsed.
  • drop_health uses a VRL condition to discard health checks and favicon requests.
  • archive writes one JSON object per line into a file per day.
  • prometheus exposes Vector's own metrics (events processed, errors, buffer sizes) for Prometheus to scrape.
  • tests is ignored at runtime and is only used by vector test.

Step 4 - Validating and testing the configuration

Validate the configuration as the vector user, so the check also covers file permissions on the data directory:

sudo -u vector vector validate /etc/vector/vector.yaml

√ Loaded ["/etc/vector/vector.yaml"]
√ Component configuration
√ Health check "archive"
√ Health check "prometheus"
------------------------------------
                           Validated

Run the unit test:

vector test /etc/vector/vector.yaml

Running tests
test parses a combined log line ... passed

If you change the VRL program later and break it, this test fails before the service is restarted. The systemd unit shipped with the package also runs vector validate before starting, so a broken configuration does not replace a running one silently.

Step 5 - Starting the service

Enable and start Vector. Restart it if it was already running, so it picks up the new group membership and configuration:

sudo systemctl enable vector
sudo systemctl restart vector
sudo systemctl status vector --no-pager

The status should show active (running). Generate some traffic against Nginx:

for i in $(seq 1 5); do curl -s -o /dev/null http://localhost/; done
curl -s -o /dev/null http://localhost/health

Watch parsed events flow through the filter in real time with vector tap, which connects to the local API:

vector tap drop_health

Each request appears as a JSON event with separate fields. The /health request does not appear, because the filter dropped it. Press Ctrl+C to stop.

Check the archive file:

tail -n 1 /var/log/vector/nginx-$(date -u +%Y-%m-%d).json

{"agent":"curl/8.5.0","client":"127.0.0.1","env":"production","file":"/var/log/nginx/access.log","host":"web01","method":"GET","path":"/","protocol":"HTTP/1.1","request":"GET / HTTP/1.1","size":615,"source_type":"file","status":200,"timestamp":"2026-03-10T13:57:02Z"}

The date in the file name comes from the event timestamp in UTC, so use date -u when looking for today's file.

Step 6 - Monitoring the pipeline

vector top shows throughput and errors for every component:

vector top

Press q to quit. The Prometheus endpoint returns the same information for your monitoring system:

curl -s http://127.0.0.1:9598/metrics | grep component_received_events_total | head -n 3

If you run Prometheus on another host, keep the exporter on localhost and scrape it through a private network or a reverse proxy rather than opening port 9598 to the Internet.

Step 7 - Shipping logs to Grafana Loki (optional)

Add a second sink that reads from the same filter. Open the configuration:

sudo nano /etc/vector/vector.yaml

Add this block under sinks:, replacing loki.example.internal with your Loki host:

  loki:
    type: loki
    inputs:
      - drop_health
    endpoint: http://loki.example.internal:3100
    encoding:
      codec: json
    labels:
      job: nginx
      host: "{{ host }}"

Keep Loki labels few and low-cardinality (job, host, environment). Fields such as path or client belong in the log line, where LogQL can filter them, not in labels.

Validate and restart:

sudo -u vector vector validate /etc/vector/vector.yaml
sudo systemctl restart vector

The validation now includes a health check against Loki, which fails if the endpoint is unreachable. In Grafana, query {job="nginx"} | json | status >= 500 to find server errors.

Troubleshooting

Permission denied on /var/log/nginx/access.log in the logs. The service was started before the vector user joined the adm group. Restart it with sudo systemctl restart vector.

No events appear. read_from: end skips existing lines, so send new requests. Check journalctl -u vector -n 50 for errors and vector top for counters that stay at zero.

Some lines are missing from the output. Lines that do not match the combined format are dropped by drop_on_error. If you use a custom log_format in Nginx, parse it with parse_regex! or switch Nginx to a JSON log format and use parse_json!.

The service does not start after a change. The unit validates the configuration first. Run sudo -u vector vector validate /etc/vector/vector.yaml to see the exact error.

Conclusion

You installed Vector from its official repository and built a tested pipeline that turns raw Nginx access logs into structured JSON, filters out noise and stores or forwards the result. From here you can add a journald source for system logs, use a route transform to send errors to a different sink, or collect host metrics with the host_metrics source and expose them next to Vector's own metrics.