Elasticsearch stores and searches large volumes of log data, and Logstash collects, parses and enriches logs before sending them to Elasticsearch. Together with Kibana they form the Elastic (ELK) stack. In this tutorial you will install Elasticsearch 9 and Logstash on a single Ubuntu 24.04 server from Elastic's official repository, keep the security features that Elasticsearch enables by default, and build a Logstash pipeline that parses Nginx access logs and indexes them.
Prerequisites
To follow this guide you need:
- A server running Ubuntu 24.04 LTS with at least 8 GB of RAM and 2 vCPUs, for example a CubePath VPS. Both services run on the JVM; with 4 GB they start, but leave little room for data.
- A non-root user with
sudoprivileges. - Nginx installed and serving some traffic, so there are entries in
/var/log/nginx/access.logto index. Any other log works if you adapt the pipeline in Step 6.
You do not need to install Java: both packages include their own supported JDK.
Step 1 - Adding the Elastic repository
Install the tools needed to fetch the repository key:
sudo apt update
sudo apt install curl gpg
Download Elastic's signing key and store it in /etc/apt/keyrings, the standard location for third-party repository keys:
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /etc/apt/keyrings/elasticsearch.gpg
Add the repository for the 9.x release series, restricted to that key:
echo "deb [signed-by=/etc/apt/keyrings/elasticsearch.gpg] https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-9.x.list
sudo apt update
Confirm that apt sees the packages from Elastic:
apt-cache policy elasticsearch logstash | grep -E '^[a-z]|Candidate'
elasticsearch:
Candidate: 9.1.4
logstash:
Candidate: 1:9.1.4-1
Your version numbers will be newer; what matters is that both packages have a candidate. Elasticsearch and Logstash must use the same major version.
Step 2 - Installing Elasticsearch
Install the package:
sudo apt install elasticsearch
On installation, Elasticsearch configures its security automatically: it generates TLS certificates for the HTTP and transport layers, enables authentication and prints a password for the elastic superuser. Look for this block in the output and save the password somewhere safe:
--------------------------- Security autoconfiguration information ------------------------------
Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.
The generated password for the elastic built-in superuser is : your_generated_password
...
If you lose it, you can set a new one at any time:
sudo /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic -i
By default, Elasticsearch only listens on localhost, which is what you want for a single-server setup: Logstash connects locally and nothing is exposed to the internet. Leave network.host in /etc/elasticsearch/elasticsearch.yml unset.
Step 3 - Setting the JVM heap size (optional)
Elasticsearch sizes its heap automatically based on the total RAM, assuming it has the machine to itself. Since Logstash runs on the same server, cap the heap explicitly. On an 8 GB server, 3 GB is a reasonable value. Create an options file:
sudo nano /etc/elasticsearch/jvm.options.d/heap.options
-Xms3g
-Xmx3g
Always set the minimum and maximum to the same value. Logstash uses a 1 GB heap by default (/etc/logstash/jvm.options), which is enough for this guide.
Step 4 - Starting and testing Elasticsearch
Enable and start the service. The first start can take up to a minute:
sudo systemctl daemon-reload
sudo systemctl enable --now elasticsearch
Copy the CA certificate that signed the HTTP certificate to your home directory, so you can use curl against the API without sudo:
sudo cp /etc/elasticsearch/certs/http_ca.crt ~/http_ca.crt
sudo chown "$USER": ~/http_ca.crt
Query the cluster with the elastic user. curl prompts for the password:
curl --cacert ~/http_ca.crt -u elastic https://localhost:9200
{
"name" : "your-hostname",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "9.1.4",
...
},
"tagline" : "You Know, for Search"
}
A response with the version number confirms that Elasticsearch is running, TLS works, and authentication is enforced. If you get Connection refused, the service is still starting or failed; check sudo journalctl -u elasticsearch -n 50.
A single-node cluster cannot place replica shards, so new indices would stay in yellow health. Create an index template that disables replicas for the Nginx indices you will create:
curl --cacert ~/http_ca.crt -u elastic -X PUT "https://localhost:9200/_index_template/nginx-access" \
-H 'Content-Type: application/json' -d '
{
"index_patterns": ["nginx-access-*"],
"template": {
"settings": { "number_of_replicas": 0 }
}
}'
{"acknowledged":true}
Step 5 - Creating a user for Logstash
Logstash should not use the elastic superuser. Create a role that can only write to the nginx-access-* indices, and a user with that role. Replace your_logstash_password with a strong password:
curl --cacert ~/http_ca.crt -u elastic -X POST "https://localhost:9200/_security/role/logstash_writer" \
-H 'Content-Type: application/json' -d '
{
"cluster": ["monitor", "manage_index_templates", "manage_ilm"],
"indices": [
{
"names": ["nginx-access-*"],
"privileges": ["write", "create", "create_index", "manage", "manage_ilm"]
}
]
}'
curl --cacert ~/http_ca.crt -u elastic -X POST "https://localhost:9200/_security/user/logstash_internal" \
-H 'Content-Type: application/json' -d '
{
"password": "your_logstash_password",
"roles": ["logstash_writer"],
"full_name": "Logstash writer"
}'
Both calls return {"role":{"created":true}} and {"created":true} respectively. Verify that the new user can authenticate:
curl --cacert ~/http_ca.crt -u logstash_internal "https://localhost:9200/_security/_authenticate?pretty"
The response lists "username" : "logstash_internal" and the logstash_writer role.
Step 6 - Installing Logstash and building the pipeline
Install Logstash from the same repository:
sudo apt install logstash
Logstash needs the Elasticsearch CA certificate to verify the TLS connection. Copy it into the Logstash configuration directory:
sudo mkdir -p /etc/logstash/certs
sudo cp /etc/elasticsearch/certs/http_ca.crt /etc/logstash/certs/
sudo chmod 0644 /etc/logstash/certs/http_ca.crt
Store the logstash_internal password in the Logstash keystore instead of writing it in the pipeline file. Create the keystore (answer y when asked about continuing without a keystore password) and add the password under the name ES_PWD:
sudo /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash create
sudo /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash add ES_PWD
The keystore is created by root, so give the logstash group read access:
sudo chown root:logstash /etc/logstash/logstash.keystore
sudo chmod 0640 /etc/logstash/logstash.keystore
Nginx logs are owned by the adm group. Add the logstash user to it so the service can read them:
sudo usermod -aG adm logstash
Now create the pipeline. Every .conf file in /etc/logstash/conf.d/ is loaded into the main pipeline:
sudo nano /etc/logstash/conf.d/nginx.conf
input {
file {
path => "/var/log/nginx/access.log"
start_position => "beginning"
}
}
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
date {
match => [ "timestamp", "dd/MMM/yyyy:HH:mm:ss Z" ]
remove_field => [ "timestamp" ]
}
}
output {
elasticsearch {
hosts => ["https://localhost:9200"]
ssl_certificate_authorities => ["/etc/logstash/certs/http_ca.crt"]
user => "logstash_internal"
password => "${ES_PWD}"
index => "nginx-access-%{+YYYY.MM.dd}"
}
}
The three sections do the following:
- input: reads
access.logfrom the beginning the first time, then follows new lines. Logstash remembers its position between restarts. - filter:
groksplits each line of the default Nginxcombinedformat into structured fields (client address, method, URL, status code, bytes, user agent), anddateuses the request time as the event timestamp instead of the time Logstash read the line. - output: sends the events over TLS to a daily index such as
nginx-access-2026.09.25, authenticated aslogstash_internal.
Test the configuration as the logstash user before starting the service:
sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash --config.test_and_exit
...
Configuration OK
[INFO ][logstash.runner] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash
Step 7 - Starting Logstash and verifying the data
Enable and start Logstash:
sudo systemctl enable --now logstash
Logstash takes 30 to 60 seconds to start. Follow its log until the pipeline is running:
sudo journalctl -u logstash -f
[INFO ][logstash.javapipeline][main] Pipeline started {"pipeline.id"=>"main"}
[INFO ][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
Press Ctrl+C to stop following the log. Generate a few requests to Nginx, then list the new indices:
curl -s http://localhost/ > /dev/null
curl --cacert ~/http_ca.crt -u elastic "https://localhost:9200/_cat/indices/nginx-access-*?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open nginx-access-2026.09.25 k3Jd0x5bQ2uYzv4fW1aZ9Q 1 0 128 0 142.3kb 142.3kb
The index is green because the template removed replicas. Look at one parsed document to confirm the fields were extracted:
curl --cacert ~/http_ca.crt -u elastic "https://localhost:9200/nginx-access-*/_search?size=1&pretty&filter_path=hits.hits._source.source,hits.hits._source.http,hits.hits._source.url"
{
"hits" : {
"hits" : [
{
"_source" : {
"source" : { "address" : "127.0.0.1" },
"http" : {
"request" : { "method" : "GET" },
"response" : { "status_code" : 200, "body" : { "bytes" : 615 } },
"version" : "1.1"
},
"url" : { "original" : "/" }
}
}
]
}
}
Field names follow the Elastic Common Schema (ECS), which Logstash uses by default, so they work out of the box with Kibana dashboards and queries.
Troubleshooting
Elasticsearch fails to start: read sudo journalctl -u elasticsearch -n 50 and /var/log/elasticsearch/elasticsearch.log. A heap larger than the available RAM, or a typo in elasticsearch.yml, are the most common causes.
Logstash logs 401 or security_exception: the password in the keystore does not match the logstash_internal user. Run the add ES_PWD command again (it asks to overwrite) and restart Logstash.
Logstash logs PKIX path building failed or certificate errors: the ssl_certificate_authorities path is wrong or not readable by the logstash user.
No documents arrive and no errors appear: the logstash user cannot read the log file. Confirm the group membership with id logstash and restart the service after adding it to adm, since group changes only apply to new processes.
Documents have a _grokparsefailure tag: the log format does not match COMBINEDAPACHELOG. This happens if you customized log_format in Nginx; adapt the grok pattern to your format.
Conclusion
You now have Elasticsearch 9 and Logstash running on Ubuntu 24.04 with TLS and authentication enabled, a least-privilege user for Logstash, and a pipeline that turns raw Nginx access logs into structured, searchable documents. Next, install Kibana from the same repository to explore the data visually, add an index lifecycle policy to delete old daily indices, and use Filebeat or Elastic Agent to ship logs from other servers to this Logstash instance.