Kyverno is a policy engine for Kubernetes that runs as an admission controller: every resource sent to the API server passes through it, and it can reject, modify or trigger the creation of other resources based on policies written in plain YAML. In this tutorial you will install Kyverno with Helm, write a validation policy that requires resource limits, a mutation policy that adds default labels, and a generation policy that creates a default-deny NetworkPolicy in every new namespace. You will finish by reading the policy reports Kyverno produces.
Prerequisites
To follow this guide you need:
- A Kubernetes cluster running a version supported by the current Kyverno release (check the compatibility table in the Kyverno documentation), for example a CubePath managed Kubernetes cluster.
kubectlconfigured with cluster-admin rights on that cluster.- Helm 3 installed on the machine where you run
kubectl. - For the generation step to have an effect, a CNI plugin that enforces NetworkPolicies (Cilium or Calico, for example).
Use a test or staging cluster the first time. An Enforce policy with a mistake can block deployments cluster-wide.
Step 1 - Installing Kyverno with Helm
Add the official Helm repository and refresh the index:
helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
Install Kyverno into its own namespace:
helm install kyverno kyverno/kyverno --namespace kyverno --create-namespace
This deploys four controllers with one replica each, which is fine for testing. On production clusters run the admission controller with at least three replicas so that a node failure does not stop the API server from admitting resources:
helm upgrade kyverno kyverno/kyverno --namespace kyverno \
--set admissionController.replicas=3 \
--set backgroundController.replicas=2 \
--set cleanupController.replicas=2 \
--set reportsController.replicas=2
Check that all the controllers are running:
kubectl -n kyverno get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
kyverno-admission-controller 1/1 1 1 60s
kyverno-background-controller 1/1 1 1 60s
kyverno-cleanup-controller 1/1 1 1 60s
kyverno-reports-controller 1/1 1 1 60s
The admission controller handles validation and mutation, the background controller handles generation and mutation of existing resources, the reports controller builds policy reports and the cleanup controller runs cleanup policies.
Create a namespace to test your policies in:
kubectl create namespace policy-test
Step 2 - Writing a validation policy
Validation policies decide whether a resource is accepted. Each rule has a failureAction: Audit lets the resource in and records a violation in a report, Enforce rejects it. Starting with Audit shows you what would break before anything is blocked.
Create a policy that requires CPU and memory requests and a memory limit on every container:
nano require-requests-limits.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-requests-limits
annotations:
policies.kyverno.io/title: Require requests and limits
spec:
background: true
rules:
- name: check-container-resources
match:
any:
- resources:
kinds:
- Pod
exclude:
any:
- resources:
namespaces:
- kube-system
- kyverno
validate:
failureAction: Audit
message: "CPU and memory requests and a memory limit are required for every container."
pattern:
spec:
containers:
- resources:
requests:
cpu: "?*"
memory: "?*"
limits:
memory: "?*"
In a pattern, ?* means "any non-empty value". background: true makes Kyverno also evaluate resources that already exist, not only new ones. Although the rule matches Pods, Kyverno automatically generates equivalent rules for Deployments, StatefulSets, Jobs and other controllers, so a bad Deployment is caught before it creates Pods.
Apply the policy and wait until it is ready:
kubectl apply -f require-requests-limits.yaml
kubectl get clusterpolicy require-requests-limits
NAME ADMISSION BACKGROUND READY AGE MESSAGE
require-requests-limits true true True 5s Ready
Create a Pod without resources. In Audit mode it is accepted, and the violation is only recorded in a policy report (you will read it in Step 5):
kubectl -n policy-test run no-limits --image=nginx:alpine
pod/no-limits created
Now switch the rule to Enforce. Edit the file and change the line under validate:
failureAction: Enforce
Apply it again and try to create another Pod without resources:
kubectl apply -f require-requests-limits.yaml
kubectl -n policy-test run no-limits-2 --image=nginx:alpine
Error from server: admission webhook "validate.kyverno.svc-fail" denied the request:
resource Pod/policy-test/no-limits-2 was blocked due to the following policies
require-requests-limits:
check-container-resources: 'validation error: CPU and memory requests and a memory limit are required for every container. rule check-container-resources failed at path /spec/containers/0/resources/limits/'
A compliant Pod is still admitted:
kubectl -n policy-test run with-limits --image=nginx:alpine \
--overrides='{"spec":{"containers":[{"name":"with-limits","image":"nginx:alpine","resources":{"requests":{"cpu":"50m","memory":"64Mi"},"limits":{"memory":"128Mi"}}}]}}'
pod/with-limits created
Step 3 - Writing a mutation policy
Mutation policies change resources as they are admitted. They are useful for defaults that nobody should have to remember. This policy adds a team label to Deployments that do not already have one, and a managed-by label on all of them:
nano add-default-labels.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: add-default-labels
spec:
rules:
- name: add-labels
match:
any:
- resources:
kinds:
- Deployment
mutate:
patchStrategicMerge:
metadata:
labels:
+(team): unassigned
managed-by: kyverno
The +( ) anchor means "add this field only if it is not already set", so a Deployment that declares its own team label keeps it. Apply the policy and create a Deployment:
kubectl apply -f add-default-labels.yaml
kubectl -n policy-test create deployment web --image=nginx:alpine
The Deployment has no resources, so if the policy from Step 2 is still in Enforce mode its Pods will be rejected; the Deployment object itself is created. Check its labels:
kubectl -n policy-test get deployment web --show-labels
NAME READY UP-TO-DATE AVAILABLE AGE LABELS
web 0/1 0 0 10s app=web,managed-by=kyverno,team=unassigned
The 0/1 confirms that the validation policy is doing its job for workloads created through a Deployment.
Step 4 - Writing a generation policy
Generation policies create new resources when a trigger resource appears. A common use is giving every new namespace a default-deny NetworkPolicy, so that traffic has to be allowed explicitly.
Kyverno needs permission to create the generated resource type. Grant it through a ClusterRole with the aggregation labels that Kyverno's own roles collect:
nano kyverno-networkpolicy-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: kyverno-manage-networkpolicies
labels:
rbac.kyverno.io/aggregate-to-admission-controller: "true"
rbac.kyverno.io/aggregate-to-background-controller: "true"
rules:
- apiGroups:
- networking.k8s.io
resources:
- networkpolicies
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
Now create the policy:
nano default-deny-networkpolicy.yaml
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: default-deny-networkpolicy
spec:
rules:
- name: generate-default-deny
match:
any:
- resources:
kinds:
- Namespace
exclude:
any:
- resources:
names:
- kube-system
- kube-public
- kube-node-lease
- kyverno
generate:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
name: default-deny-all
namespace: "{{request.object.metadata.name}}"
synchronize: true
data:
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
{{request.object.metadata.name}} is replaced with the name of the namespace being created. synchronize: true makes Kyverno recreate the NetworkPolicy if someone deletes it and update it if you change the policy.
Apply both files and create a new namespace:
kubectl apply -f kyverno-networkpolicy-rbac.yaml
kubectl apply -f default-deny-networkpolicy.yaml
kubectl create namespace team-a
Check that the NetworkPolicy was generated:
kubectl -n team-a get networkpolicy
NAME POD-SELECTOR AGE
default-deny-all <none> 5s
WarningA default-deny policy also blocks DNS. Teams deploying into these namespaces must add NetworkPolicies that allow egress to CoreDNS and to the services they use.
Step 5 - Reading policy reports
For policies with background: true, Kyverno periodically re-evaluates existing resources and stores the results in PolicyReport objects (one per resource, in the resource's namespace) and ClusterPolicyReport objects for cluster-scoped resources.
List the reports in the test namespace:
kubectl -n policy-test get policyreport
NAME KIND NAME PASS FAIL WARN ERROR SKIP AGE
0b1e6a7c-6f1f-4a73-9d59-2f1e4c8d7a11 Pod no-limits 0 1 0 0 0 2m
5c2d8e90-1b4a-4e7e-8c1f-6a3b9d0e2f44 Pod with-limits 1 0 0 0 0 2m
9f7a3c21-8d6e-4b2a-a1c5-7e0d4b3f6a88 Deployment web 0 1 0 0 0 1m
The no-limits Pod created in Audit mode shows up as a failure. Across the whole cluster, list only the failing results with the policy and resource involved:
kubectl get policyreport -A -o json | jq -r '.items[] | .scope as $s | .results[]? | select(.result=="fail") | "\($s.namespace)/\($s.name)\t\(.policy)\t\(.message)"'
This requires jq (sudo apt install jq). Reviewing these reports is the safe way to roll out a new policy: deploy it in Audit, fix or exempt the failing workloads, then switch to Enforce.
Step 6 - Adding the Pod Security policy set
Kyverno publishes a Helm chart with ready-made policies that implement the Kubernetes Pod Security Standards. Install the baseline profile, which blocks privileged containers, host namespaces and other dangerous settings:
helm install kyverno-policies kyverno/kyverno-policies \
--namespace kyverno \
--set podSecurityStandard=baseline
List the installed policies:
kubectl get clusterpolicy
The chart's policies start in Audit mode. Watch their reports for a while, then move to the stricter restricted profile or to Enforce once your workloads comply.
Troubleshooting
A policy stays READY False: describe it with kubectl describe clusterpolicy <name> and read the status conditions. Invalid patterns, unknown fields or missing RBAC for generated resources are reported there.
A policy does not seem to apply: check the admission controller logs with kubectl -n kyverno logs deploy/kyverno-admission-controller --tail=50, and confirm that the resource's namespace is not listed under exclude. Test without creating anything by adding --dry-run=server to kubectl apply, which still passes through the webhooks.
Generated resources are not created: look at kubectl -n kyverno logs deploy/kyverno-background-controller for permission errors. The ClusterRole from Step 4 must carry both aggregation labels.
The cluster rejects everything after Kyverno goes down: by default Kyverno's webhooks fail closed for Enforce policies. Run several admission controller replicas in production, and if you need an emergency exit, switch the affected policies to Audit or delete them.
Conclusion
You installed Kyverno, used a validation policy in Audit and Enforce modes, added defaults with a mutation policy, generated a NetworkPolicy for every new namespace and read the resulting policy reports. As next steps, verify container image signatures at admission time with Kyverno's verifyImages rules, store your policies in Git and apply them through your GitOps tool, and explore the community policy library for ready-made rules.