Ingress Configuration in Kubernetes: Complete Production Guide

Kubernetes Ingress provides HTTP and HTTPS routing to services within a cluster, offering load balancing, SSL/TLS termination, and name-based virtual hosting. This comprehensive guide covers Ingress controllers, configuration, TLS setup, and production best practices.

Introduction

Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Unlike LoadBalancer services that require a cloud provider's load balancer for each service, a single Ingress can route traffic to multiple services based on rules.

Why Use Ingress?

  • Cost-Effective: Single load balancer for multiple services
  • Flexible Routing: Path and host-based routing
  • SSL/TLS Termination: Centralized certificate management
  • Load Balancing: Distribute traffic across pods
  • Advanced Features: Rate limiting, authentication, rewrites

Ingress vs LoadBalancer vs NodePort

FeatureIngressLoadBalancerNodePort
CostLowHigh (per service)Free
L7 RoutingYesNoNo
SSL TerminationYesNoNo
External IP1 for many services1 per serviceNode IPs

Prerequisites

  • Kubernetes cluster (1.19+)
  • kubectl configured
  • Domain name (for production)
  • Basic understanding of Services and DNS

Verify cluster:

kubectl version --client
kubectl cluster-info
kubectl get nodes

Ingress Concepts

Ingress Architecture

Internet
    ↓
Ingress Controller (LoadBalancer Service)
    ↓
Ingress Resource (routing rules)
    ↓
Backend Services
    ↓
Pods

Ingress Components

  • Ingress Controller: Implementation (NGINX, Traefik, HAProxy, etc.)
  • Ingress Resource: Routing rules defined in YAML
  • Backend Services: Target services
  • TLS Secrets: SSL/TLS certificates

Installing Ingress Controllers

# Install via Helm
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace

# Or via manifest
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

# Verify installation
kubectl get pods -n ingress-nginx
kubectl get svc -n ingress-nginx

Traefik Ingress Controller

# Install via Helm
helm repo add traefik https://traefik.github.io/charts
helm repo update

helm install traefik traefik/traefik \
  --namespace traefik \
  --create-namespace

# Verify
kubectl get pods -n traefik

HAProxy Ingress Controller

# Install via Helm
helm repo add haproxytech https://haproxytech.github.io/helm-charts
helm repo update

helm install haproxy haproxytech/kubernetes-ingress \
  --namespace haproxy-ingress \
  --create-namespace

# Verify
kubectl get pods -n haproxy-ingress

Get External IP

# Get LoadBalancer IP
kubectl get svc -n ingress-nginx ingress-nginx-controller

# Wait for EXTERNAL-IP
NAME                                 TYPE           EXTERNAL-IP
ingress-nginx-controller             LoadBalancer   34.123.45.67

Basic Ingress Configuration

Simple Ingress Example

# ingress-basic.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: simple-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

# Create backend service first
kubectl create deployment web --image=nginx:alpine
kubectl expose deployment web --port=80 --target-port=80

# Apply ingress
kubectl apply -f ingress-basic.yaml

# Verify
kubectl get ingress
kubectl describe ingress simple-ingress

Complete Example with Service

# web-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: web-service
spec:
  selector:
    app: web
  ports:
  - port: 80
    targetPort: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-ingress
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

# Apply all resources
kubectl apply -f web-deployment.yaml

# Test
curl http://<EXTERNAL-IP>/

Path-Based Routing

Route traffic based on URL path to different services.

Path-Based Ingress

# ingress-paths.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: path-based-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /api(/|$)(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: api-service
            port:
              number: 8080

      - path: /web(/|$)(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: web-service
            port:
              number: 80

      - path: /
        pathType: Prefix
        backend:
          service:
            name: default-service
            port:
              number: 80

PathType Options

  • Exact: Exact URL match
  • Prefix: Match based on URL path prefix
  • ImplementationSpecific: Controller-dependent behavior
# pathtype-examples.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: pathtype-example
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      # Exact match only /api
      - path: /api
        pathType: Exact
        backend:
          service:
            name: api-service
            port:
              number: 8080

      # Matches /app, /app/, /app/test
      - path: /app
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 3000

Host-Based Routing

Route traffic based on hostname to different services.

Single Host Ingress

# ingress-host.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: host-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Multiple Hosts Ingress

# ingress-multi-host.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-host-ingress
spec:
  ingressClassName: nginx
  rules:
  # Main website
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

  # API subdomain
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

  # Admin subdomain
  - host: admin.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: admin-service
            port:
              number: 3000

Wildcard Hosts

# ingress-wildcard.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wildcard-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: "*.example.com"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: catch-all-service
            port:
              number: 80

TLS/SSL Configuration

Creating TLS Secret

# Generate self-signed certificate (testing only)
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout tls.key -out tls.crt \
  -subj "/CN=example.com/O=example.com"

# Create secret from files
kubectl create secret tls example-tls \
  --cert=tls.crt \
  --key=tls.key

# Or from cert-manager (Let's Encrypt)
# Will cover below

Ingress with TLS

# ingress-tls.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-ingress
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - example.com
    - www.example.com
    secretName: example-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80
  - host: www.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

# Apply
kubectl apply -f ingress-tls.yaml

# Test HTTPS
curl https://example.com

Automatic TLS with cert-manager

# Install cert-manager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml

# Create ClusterIssuer for Let's Encrypt
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

# ingress-certmanager.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: certmanager-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - example.com
    secretName: example-tls-auto
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Advanced Configurations

Custom Annotations

# ingress-annotations.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: advanced-ingress
  annotations:
    # URL rewriting
    nginx.ingress.kubernetes.io/rewrite-target: /$2

    # SSL redirect
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"

    # Rate limiting
    nginx.ingress.kubernetes.io/limit-rps: "10"

    # CORS
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, OPTIONS"

    # Custom timeouts
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "30"

    # Client body size
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
spec:
  ingressClassName: nginx
  rules:
  - host: example.com
    http:
      paths:
      - path: /api(/|$)(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: api-service
            port:
              number: 8080

Basic Authentication

# Create htpasswd file
htpasswd -c auth username

# Create secret
kubectl create secret generic basic-auth --from-file=auth

# ingress-basic-auth.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: auth-ingress
  annotations:
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required'
spec:
  ingressClassName: nginx
  rules:
  - host: admin.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: admin-service
            port:
              number: 80

Rate Limiting

# ingress-rate-limit.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rate-limited-ingress
  annotations:
    nginx.ingress.kubernetes.io/limit-rps: "10"
    nginx.ingress.kubernetes.io/limit-connections: "5"
    nginx.ingress.kubernetes.io/limit-rpm: "100"
spec:
  ingressClassName: nginx
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 8080

URL Redirects

# ingress-redirect.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: redirect-ingress
  annotations:
    # Redirect www to non-www
    nginx.ingress.kubernetes.io/permanent-redirect: "https://example.com$request_uri"
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80

Production Best Practices

High Availability

# ha-ingress-controller.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  replicas: 3  # Multiple replicas
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app.kubernetes.io/name
                  operator: In
                  values:
                  - ingress-nginx
              topologyKey: kubernetes.io/hostname

Resource Limits

spec:
  template:
    spec:
      containers:
      - name: controller
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
          limits:
            cpu: 1000m
            memory: 1Gi

Monitoring and Logging

metadata:
  annotations:
    # Prometheus metrics
    prometheus.io/scrape: "true"
    prometheus.io/port: "10254"

    # Custom logging
    nginx.ingress.kubernetes.io/configuration-snippet: |
      access_log /var/log/nginx/custom-access.log custom;

Security Headers

metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      more_set_headers "X-Frame-Options: DENY";
      more_set_headers "X-Content-Type-Options: nosniff";
      more_set_headers "X-XSS-Protection: 1; mode=block";
      more_set_headers "Strict-Transport-Security: max-age=31536000; includeSubDomains";

Troubleshooting

Common Issues

# Check Ingress status
kubectl get ingress
kubectl describe ingress <ingress-name>

# Check Ingress Controller logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

# Check backend endpoints
kubectl get endpoints <service-name>

# Test from within cluster
kubectl run curl --image=curlimages/curl --rm -it -- curl http://service-name

# Check TLS secret
kubectl get secret <secret-name> -o yaml
kubectl describe secret <secret-name>

Debug Mode

# Enable debug logging
kubectl edit deployment -n ingress-nginx ingress-nginx-controller

# Add to args:
- --v=3  # Increase verbosity

Certificate Issues

# Check cert-manager
kubectl get certificates
kubectl get certificaterequests
kubectl describe certificate <cert-name>

# Force renewal
kubectl delete secret <tls-secret>

Conclusion

Ingress provides powerful HTTP/HTTPS routing capabilities for Kubernetes clusters, enabling cost-effective, flexible traffic management with advanced features like TLS termination and authentication.

Key Takeaways

  • Ingress Controllers: Choose appropriate controller (NGINX, Traefik, etc.)
  • Routing: Path and host-based routing flexibility
  • TLS/SSL: Centralized certificate management
  • Annotations: Controller-specific advanced features
  • Production: HA, monitoring, security headers

Quick Reference

# Installation
helm install ingress-nginx ingress-nginx/ingress-nginx

# Basic ingress
kubectl create ingress simple --rule="example.com/*=web-service:80"

# With TLS
kubectl create ingress tls-ingress \
  --rule="example.com/*=web-service:80,tls=example-tls"

# Operations
kubectl get ingress
kubectl describe ingress <name>
kubectl edit ingress <name>
kubectl delete ingress <name>

Next Steps

  1. Deploy: Set up Ingress controller in your cluster
  2. Configure: Create Ingress rules for your services
  3. Secure: Implement TLS with cert-manager
  4. Monitor: Set up metrics and alerting
  5. Optimize: Fine-tune annotations and caching
  6. Scale: Configure HA for production
  7. Advanced: Implement auth, rate limiting, WAF

Ingress is essential for production Kubernetes deployments. Master it to build robust, scalable applications!