CI/CD Pipeline Security: Secrets, Scanning and Signing

A CI/CD pipeline has write access to your source code, your registries and your production servers, which makes it one of the most valuable targets in your infrastructure. Most real-world pipeline compromises come from a few causes: leaked long-lived credentials, untrusted code running on privileged runners, and dependencies or images that nobody checked. This guide explains the controls that address each of those causes and shows how to implement them with concrete GitLab CI jobs, using open source tools: Gitleaks for secret scanning, Trivy for vulnerability scanning and SBOMs, and Cosign for image signing.

Prerequisites

The examples assume:

• A project on GitLab (GitLab.com or self-managed) with at least the Maintainer role, and a runner with the Docker executor. The same ideas apply to GitHub Actions, Jenkins or any other CI system.
• A container registry the pipeline pushes to, such as the GitLab Container Registry or a self-hosted Harbor.
• For the local commands, a workstation or server with Docker installed.

Threats and controls at a glance

The sections below cover each control in the order a pipeline runs.

Protecting the pipeline definition and branches

The pipeline file is code that runs with your credentials, so treat changes to it like changes to production.

In GitLab, go to Settings > Repository > Protected branches and protect main so that nobody can push directly and only Maintainers can merge. Then require at least one approval on merge requests under Settings > Merge requests. To require extra review for pipeline changes, add a CODEOWNERS file at the root of the repository and enable Require approval from code owners for the protected branch:

# CODEOWNERS
/.gitlab-ci.yml   @your_group/platform-team
/deploy/          @your_group/platform-team

Signed commits let reviewers and GitLab verify who authored a change. Git supports signing with an existing SSH key, which is simpler than managing GPG keys:

git config --global gpg.format ssh
git config --global user.signingkey ~/.ssh/id_ed25519.pub
git config --global commit.gpgsign true

Add the same public key to your GitLab profile as a Signing key under Preferences > SSH Keys. Your commits then show a Verified badge. On GitLab Premium and Ultimate you can use a push rule to reject unsigned commits.

Managing secrets

Store secrets as protected, masked variables

Never write credentials in .gitlab-ci.yml or in the repository. Store them under Settings > CI/CD > Variables with these options:

• Protected: the variable is only available to pipelines on protected branches and tags. A merge request from a feature branch cannot read your production deploy key.
• Masked (or Masked and hidden): GitLab replaces the value with [MASKED] in job logs. Masking is a safety net, not a guarantee; a script can still leak a secret by transforming it, for example by encoding it in base64.
• Type: File for certificates, kubeconfigs and SSH keys. GitLab writes the value to a temporary file and puts the file path in the variable, so the secret never appears on a command line.

Use a secret in a job without echoing it or passing it as a command line argument, where it could show up in process lists:

deploy:
  stage: deploy
  image: alpine:3.20
  environment: production
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  before_script:
    - apk add --no-cache openssh-client
    - install -m 600 "$DEPLOY_SSH_KEY" ~/.ssh_deploy_key
  script:
    - ssh -i ~/.ssh_deploy_key -o StrictHostKeyChecking=yes deploy@your_server_ip "sudo systemctl restart myapp"

Here DEPLOY_SSH_KEY is a protected variable of type File. You also need a known_hosts entry for your_server_ip (for example from another File variable) so that host key checking can stay enabled.

Prefer short-lived credentials over static keys

A static cloud key stored in a CI variable stays valid until someone rotates it. With OpenID Connect (OIDC), each job receives a signed token that identifies the project, branch and environment, and exchanges it for credentials that expire in minutes. GitLab issues these tokens with the id_tokens keyword:

deploy:
  stage: deploy
  id_tokens:
    VAULT_ID_TOKEN:
      aud: https://vault.your_domain
  script:
    - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=myapp-production jwt="$VAULT_ID_TOKEN")"
    - vault kv get -field=password secret/myapp/database

On the Vault side (or AWS, GCP, Azure), you configure a role that only trusts tokens whose claims match, for example project_path equal to your project and ref_protected equal to true. A leaked token is useless a few minutes later, and a job from an unprotected branch cannot assume the production role at all.

Scan for committed secrets

Secrets still end up in Git by accident. Gitleaks detects API keys, private keys and tokens in the repository and its history. Run it in the pipeline so a leaked secret is caught before merge:

secret-scan:
  stage: test
  image:
    name: zricethezav/gitleaks:v8.21.2
    entrypoint: [""]
  variables:
    GIT_DEPTH: "0"
  script:
    - gitleaks git --verbose --redact .

--redact hides the secret value in the output, so the scan itself does not leak it into the job log. The job fails with a non-zero exit code when it finds something:

Finding:     aws_secret_access_key = REDACTED
RuleID:      aws-access-token
File:        config/settings.py
Line:        12
Commit:      4f2c1a9...

WRN leaks found: 1

Important

If a real secret was committed, removing it from the repository is not enough. Revoke and rotate it immediately; anyone who cloned or forked the repository still has it.

Controlling where code runs

Runners are shared infrastructure, and a job can read anything its runner can reach. Separate them by trust level:

• Mark the runner that deploys to production as Protected in Settings > CI/CD > Runners. A protected runner only runs jobs from protected branches and tags, so code from an unreviewed merge request never reaches it.
• Use protected environments (Settings > CI/CD > Protected environments, available on Premium) to restrict who can deploy to production and to require an approval before the job runs.
• Avoid privileged = true on runners that run untrusted code. Privileged Docker-in-Docker gives the job root access to the runner host. Build images with rootless tools such as BuildKit or Buildah where you can.
• Prefer the Docker executor with fresh containers per job over the shell executor, so one job cannot leave files or processes behind for the next.

Scanning dependencies and images

Pin dependencies

Scanning only works if the pipeline installs exactly what was reviewed. Commit lockfiles and install from them:

• Node.js: commit package-lock.json and use npm ci, which fails if the lockfile and package.json disagree.
• Python: pin versions with hashes in requirements.txt and install with pip install --require-hashes -r requirements.txt.
• Container base images and CI tool images: pin a specific version tag, or better a digest (image@sha256:...), instead of latest.

Scan with Trivy

Trivy scans a repository for vulnerable dependencies, misconfigurations and secrets, and scans built container images for vulnerable OS packages and libraries. This job scans the source tree and fails on high or critical findings:

dependency-scan:
  stage: test
  image:
    name: aquasec/trivy:0.58.1
    entrypoint: [""]
  script:
    - trivy fs --scanners vuln,misconfig --severity HIGH,CRITICAL --exit-code 1 .

After the image is built and pushed, scan the image itself. --ignore-unfixed skips vulnerabilities that have no fix yet, which keeps the job actionable:

image-scan:
  stage: scan
  image:
    name: aquasec/trivy:0.58.1
    entrypoint: [""]
  variables:
    TRIVY_USERNAME: $CI_REGISTRY_USER
    TRIVY_PASSWORD: $CI_REGISTRY_PASSWORD
  script:
    - trivy image --format cyclonedx --output sbom.cdx.json "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
    - trivy image --severity HIGH,CRITICAL --ignore-unfixed --exit-code 1 "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
  artifacts:
    when: always
    paths:
      - sbom.cdx.json

The output lists each finding with the installed and fixed versions:

Total: 2 (HIGH: 2, CRITICAL: 0)

Library   Vulnerability    Severity  Installed Version  Fixed Version
libssl3   CVE-2024-xxxxx   HIGH      3.0.13-r0          3.0.15-r0

Replace the Trivy version with the current release. When a finding is accepted after review, list its CVE ID in a .trivyignore file in the repository with a comment explaining why, instead of lowering the severity threshold for everything.

Generate an SBOM

A software bill of materials (SBOM) lists every package inside an image. When the next major CVE is announced, you can check which images contain the affected package without rescanning everything. Trivy can produce a CycloneDX SBOM:

trivy image --format cyclonedx --output sbom.cdx.json registry.your_domain/myapp/web:1.4.2

The image-scan job above already generates sbom.cdx.json and keeps it as a job artifact. The next section attaches it to the image as a signed attestation.

Signing and verifying images

Scanning tells you an image was safe when it was built. Signing proves that the image you deploy is the same one your pipeline built. Cosign, from the Sigstore project, stores signatures in the registry next to the image.

Install Cosign on your workstation from the .deb package on its releases page, then generate a key pair. Cosign asks for a password to encrypt the private key:

cosign generate-key-pair

Enter password for private key:
Enter password for private key again:
Private key written to cosign.key
Public key written to cosign.pub

Store cosign.key as a protected File variable named COSIGN_KEY and its password as a protected, masked variable named COSIGN_PASSWORD. Commit cosign.pub to the repository or distribute it to the systems that deploy.

Always sign by digest, not by tag, because a tag can be moved to a different image. The build job knows the digest right after the push, so have it export the digest to later jobs through a dotenv report:

build-image:
  stage: build
  image: docker:27
  services:
    - docker:27-dind
  variables:
    DOCKER_TLS_CERTDIR: "/certs"
  script:
    - echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
    - docker build -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" .
    - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
    - echo "IMAGE_DIGEST=$(docker inspect --format '{{index .RepoDigests 0}}' "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA")" > build.env
  artifacts:
    reports:
      dotenv: build.env

IMAGE_DIGEST then contains a reference such as registry.your_domain/myapp/web@sha256:... in every later job. The signing job installs Cosign from the Alpine repositories, signs the digest and attaches the SBOM from the image-scan artifacts as an attestation. Cosign reads the key password from the COSIGN_PASSWORD variable:

sign-image:
  stage: sign
  image: alpine:3.20
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  before_script:
    - apk add --no-cache cosign
  script:
    - echo "$CI_REGISTRY_PASSWORD" | cosign login "$CI_REGISTRY" -u "$CI_REGISTRY_USER" --password-stdin
    - cosign sign --yes --key "$COSIGN_KEY" "$IMAGE_DIGEST"
    - cosign attest --yes --key "$COSIGN_KEY" --type cyclonedx --predicate sbom.cdx.json "$IMAGE_DIGEST"

Before deploying, verify the signature with the public key. The command fails if the image is unsigned or was signed with a different key:

cosign verify --key cosign.pub registry.your_domain/myapp/web@sha256:your_image_digest

Verification for registry.your_domain/myapp/web@sha256:... --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key

In Kubernetes, an admission controller such as Kyverno or the Sigstore policy controller can run this verification automatically and reject unsigned images.

Putting it together

A hardened pipeline for a containerized application runs these stages in order:

1. test: unit tests, secret-scan with Gitleaks and dependency-scan with Trivy. Any failure stops the pipeline before an image exists.
2. build: build the image from a pinned base image and push it with a unique tag such as the commit SHA.
3. scan: image-scan with Trivy and SBOM generation.
4. sign: sign the image digest and attach the SBOM, only on the default branch.
5. deploy: verify the signature, then deploy by digest from a protected runner, using short-lived credentials and a protected environment that requires approval.

Pipelines on feature branches still run the test, build and scan stages, so developers get feedback early, but they never touch production credentials or runners.

Conclusion

Securing a pipeline comes down to three questions: who can change what runs, which credentials a job can reach, and whether you can prove what you deploy. Protected branches, runners and variables answer the first two; scanning, SBOMs and signatures answer the third. Next, you can move remaining static cloud keys to OIDC, enforce signature verification with an admission controller in your cluster, and schedule a weekly pipeline that rescans your deployed images for newly published CVEs.