Ingress Configuration in Kubernetes: Complete Production Guide

Kubernetes Ingress provides HTTP and HTTPS routing to services within a cluster, offering load balancing, SSL/TLS termination, and name-based virtual hosting. This comprehensive guide covers Ingress controllers, configuration, TLS setup, and production best practices.

Table of Contents

• Introduction
• Prerequisites
• Ingress Concepts
• Installing Ingress Controllers
• Basic Ingress Configuration
• Path-Based Routing
• Host-Based Routing
• TLS/SSL Configuration
• Advanced Configurations
• Production Best Practices
• Troubleshooting
• Conclusion

Introduction

Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Unlike LoadBalancer services that require a cloud provider's load balancer for each service, a single Ingress can route traffic to multiple services based on rules.

Why Use Ingress?

• Cost-Effective: Single load balancer for multiple services
• Flexible Routing: Path and host-based routing
• SSL/TLS Termination: Centralized certificate management
• Load Balancing: Distribute traffic across pods
• Advanced Features: Rate limiting, authentication, rewrites

Ingress vs LoadBalancer vs NodePort

Prerequisites

• Kubernetes cluster (1.19+)
• kubectl configured
• Domain name (for production)
• Basic understanding of Services and DNS

Verify cluster:

kubectl version --client
kubectl cluster-info
kubectl get nodes

Ingress Concepts

Ingress Architecture

Internet
    ↓
Ingress Controller (LoadBalancer Service)
    ↓
Ingress Resource (routing rules)
    ↓
Backend Services
    ↓
Pods

Ingress Components

• Ingress Controller: Implementation (NGINX, Traefik, HAProxy, etc.)
• Ingress Resource: Routing rules defined in YAML
• Backend Services: Target services
• TLS Secrets: SSL/TLS certificates

Installing Ingress Controllers

NGINX Ingress Controller (Most Popular)

# Install via Helm
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace

# Or via manifest
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

# Verify installation
kubectl get pods -n ingress-nginx
kubectl get svc -n ingress-nginx

Traefik Ingress Controller

# Install via Helm
helm repo add traefik https://traefik.github.io/charts
helm repo update

helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace

# Verify
kubectl get pods -n traefik

HAProxy Ingress Controller

# Install via Helm
helm repo add haproxytech https://haproxytech.github.io/helm-charts
helm repo update

helm install haproxy haproxytech/kubernetes-ingress \
  --namespace haproxy-ingress \
  --create-namespace

# Verify
kubectl get pods -n haproxy-ingress

Get External IP

# Get LoadBalancer IP
kubectl get svc -n ingress-nginx ingress-nginx-controller

# Wait for EXTERNAL-IP
NAME                                 TYPE           EXTERNAL-IP
ingress-nginx-controller             LoadBalancer   34.123.45.67

Basic Ingress Configuration

Simple Ingress Example

# ingress-basic.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: simple-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

# Create backend service first
kubectl create deployment web --image=nginx:alpine
kubectl expose deployment web --port=80 --target-port=80

# Apply ingress
kubectl apply -f ingress-basic.yaml

# Verify
kubectl get ingress
kubectl describe ingress simple-ingress

Complete Example with Service

# web-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

# Apply all resources
kubectl apply -f web-deployment.yaml

# Test
curl http://<EXTERNAL-IP>/

Path-Based Routing

Route traffic based on URL path to different services.

Path-Based Ingress

# ingress-paths.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: path-based-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /api(/|$)(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: api-service
            port:
              number: 8080

      - path: /web(/|$)(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: web-service
            port:
              number: 80

      - path: /
        pathType: Prefix
        backend:
          service:
            name: default-service
            port:
              number: 80

PathType Options

• Exact: Exact URL match
• Prefix: Match based on URL path prefix
• ImplementationSpecific: Controller-dependent behavior

# pathtype-examples.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: pathtype-example
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      # Exact match only /api
      - path: /api
        pathType: Exact
        backend:
          service:
            name: api-service
            port:
              number: 8080

      # Matches /app, /app/, /app/test
      - path: /app
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 3000

Host-Based Routing

Route traffic based on hostname to different services.

Single Host Ingress

# ingress-host.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: host-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Multiple Hosts Ingress

# ingress-multi-host.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-host-ingress
spec:
  ingressClassName: nginx
  rules:
  # Main website
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

  # API subdomain
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

  # Admin subdomain
  - host: admin.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: admin-service
            port:
              number: 3000

Wildcard Hosts

# ingress-wildcard.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wildcard-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: "*.example.com"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: catch-all-service
            port:
              number: 80

TLS/SSL Configuration

Creating TLS Secret

# Generate self-signed certificate (testing only)
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout tls.key -out tls.crt \
  -subj "/CN=example.com/O=example.com"

# Create secret from files
kubectl create secret tls example-tls \
  --cert=tls.crt \
  --key=tls.key

# Or from cert-manager (Let's Encrypt)
# Will cover below

Ingress with TLS

# ingress-tls.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-ingress
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - example.com
    - www.example.com
    secretName: example-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80
  - host: www.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

# Apply
kubectl apply -f ingress-tls.yaml

# Test HTTPS
curl https://example.com

Automatic TLS with cert-manager

# Install cert-manager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml

# Create ClusterIssuer for Let's Encrypt
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

# ingress-certmanager.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: certmanager-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - example.com
    secretName: example-tls-auto
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Advanced Configurations

Custom Annotations

# ingress-annotations.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: advanced-ingress
  annotations:
    # URL rewriting
    nginx.ingress.kubernetes.io/rewrite-target: /$2

    # SSL redirect
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"

    # Rate limiting
    nginx.ingress.kubernetes.io/limit-rps: "10"

    # CORS
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, OPTIONS"

    # Custom timeouts
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "30"

    # Client body size
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
spec:
  ingressClassName: nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /api(/|$)(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: api-service
            port:
              number: 8080

Basic Authentication

# Create htpasswd file
htpasswd -c auth username

# Create secret
kubectl create secret generic basic-auth --from-file=auth

# ingress-basic-auth.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: auth-ingress
  annotations:
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
spec:
  ingressClassName: nginx
  rules:
  - host: admin.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: admin-service
            port:
              number: 80

Rate Limiting

# ingress-rate-limit.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rate-limited-ingress
  annotations:
    nginx.ingress.kubernetes.io/limit-rps: "10"
    nginx.ingress.kubernetes.io/limit-connections: "5"
    nginx.ingress.kubernetes.io/limit-rpm: "100"
spec:
  ingressClassName: nginx
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

URL Redirects

# ingress-redirect.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: redirect-ingress
  annotations:
    # Redirect www to non-www
    nginx.ingress.kubernetes.io/permanent-redirect: "https://example.com$request_uri"
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Production Best Practices

High Availability

# ha-ingress-controller.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  replicas: 3  # Multiple replicas
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app.kubernetes.io/name
                  operator: In
                  values:
                  - ingress-nginx
              topologyKey: kubernetes.io/hostname

Resource Limits

spec:
  template:
    spec:
      containers:
      - name: controller
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
          limits:
            cpu: 1000m
            memory: 1Gi

Monitoring and Logging

metadata:
  annotations:
    # Prometheus metrics
    prometheus.io/scrape: "true"
    prometheus.io/port: "10254"

    # Custom logging
    nginx.ingress.kubernetes.io/configuration-snippet: |
      access_log /var/log/nginx/custom-access.log custom;

Security Headers

metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      more_set_headers "X-Frame-Options: DENY";
      more_set_headers "X-Content-Type-Options: nosniff";
      more_set_headers "X-XSS-Protection: 1; mode=block";
      more_set_headers "Strict-Transport-Security: max-age=31536000; includeSubDomains";

Troubleshooting

Common Issues

# Check Ingress status
kubectl get ingress
kubectl describe ingress <ingress-name>

# Check Ingress Controller logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

# Check backend endpoints
kubectl get endpoints <service-name>

# Test from within cluster
kubectl run curl --image=curlimages/curl --rm -it -- curl http://service-name

# Check TLS secret
kubectl get secret <secret-name> -o yaml
kubectl describe secret <secret-name>

Debug Mode

# Enable debug logging
kubectl edit deployment -n ingress-nginx ingress-nginx-controller

# Add to args:
- --v=3  # Increase verbosity

Certificate Issues

# Check cert-manager
kubectl get certificates
kubectl get certificaterequests
kubectl describe certificate <cert-name>

# Force renewal
kubectl delete secret <tls-secret>

Conclusion

Ingress provides powerful HTTP/HTTPS routing capabilities for Kubernetes clusters, enabling cost-effective, flexible traffic management with advanced features like TLS termination and authentication.

Key Takeaways

• Ingress Controllers: Choose appropriate controller (NGINX, Traefik, etc.)
• Routing: Path and host-based routing flexibility
• TLS/SSL: Centralized certificate management
• Annotations: Controller-specific advanced features
• Production: HA, monitoring, security headers

Quick Reference

# Installation
helm install ingress-nginx ingress-nginx/ingress-nginx

# Basic ingress
kubectl create ingress simple --rule="example.com/*=web-service:80"

# With TLS
kubectl create ingress tls-ingress \
  --rule="example.com/*=web-service:80,tls=example-tls"

# Operations
kubectl get ingress
kubectl describe ingress <name>
kubectl edit ingress <name>
kubectl delete ingress <name>

Next Steps

1. Deploy: Set up Ingress controller in your cluster
2. Configure: Create Ingress rules for your services
3. Secure: Implement TLS with cert-manager
4. Monitor: Set up metrics and alerting
5. Optimize: Fine-tune annotations and caching
6. Scale: Configure HA for production
7. Advanced: Implement auth, rate limiting, WAF

Ingress is essential for production Kubernetes deployments. Master it to build robust, scalable applications!