Handling build-time secrets in Kubernetes with Tekton and Vault

I knew this day was coming. Eventually.

My usual Kubernetes secrets workflow works like this: FluxCD deploys an ExternalSecret, External Secrets Operator (ESO) fetches credentials from Vault, Kubernetes creates a Secret, and the deployment mounts it as environment variables. Everything happens at run-time, when containers are already running and ready to consume those secrets. It’s the happy path that 90% of my CI/CD workloads follow without issue.

Static site generators like Astro, Next.js, or Nuxt don’t play by those rules. Sometimes they will need API keys during the customary npm run build to generate the HTML files (in my case, a GITHUB_TOKEN to fetch repository stats for this very website). By the time ESO can inject secrets into a deployment, the build has already failed.

This is because when we are creating our app’s image to be deployed to our container registry of choice (Harbor I choose you!) we need to tell our CI tool that some juicy secrets are coming and that they should be included in the pod that is building the image. The Kubernetes Deployment resources are not the ones to take care of that job, as we are not yet deploying the app.

So when I finally finished the Moreira.dev website, which is built using Astro, I had my first static website hungry for secrets at build-time, and I wasn’t ready.

This is how I ended up solving it, including the two approaches that (almost) didn’t work for my setup and why.

Attempt #1: Kaniko with build args

Kaniko (the Kubernetes-native image builder that doesn’t need Docker-in-Docker privileges) supports --build-arg flags, just like regular Docker builds. The plan was simple: fetch secrets in a Tekton task, format them as build arguments, pass them to Kaniko.

Here’s what I tried:

# Part of a Tekton Pipeline
- name: fetch-secrets
  runAfter: ["fetch-source"]
  taskSpec:
    results:
      - name: build-args
        type: array
    steps:
      - name: get-vault-secrets
        image: hashicorp/vault:latest
        script: |
          #!/bin/sh
          apk add --no-cache jq

          export VAULT_ADDR=http://vault.vault.svc:8200
          # This one is the direct kv route in Vault with your secrets
          SECRET_PATH="homelab/$(params.gitRepoName)/$(params.branchName)"

          # Log in to Vault
          LOGIN_JSON=$(vault write -format=json auth/kubernetes/login \
            role=tekton-pipelines \
            jwt=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token))

          export VAULT_TOKEN=$(echo $LOGIN_JSON | jq -r .auth.client_token)

          # Fetch and format as build args: ["--build-arg", "KEY=VAL", "--build-arg", "K2=V2"]
          vault kv get -format=json "$SECRET_PATH" | \
            jq -c '[.data.data | to_entries[] | "--build-arg", "\(.key)=\(.value)"]' \
            > $(results.build-args.path)

          # Fallback for empty results
          if [ ! -s $(results.build-args.path) ]; then
            echo "[]" > $(results.build-args.path)
          fi

- name: build-and-push
  runAfter: ["fetch-secrets"]
  taskRef:
    resolver: hub
    params:
      - { name: name, value: kaniko }
      - { name: kind, value: task }
      - { name: version, value: "0.7.0" }
  params:
    - name: IMAGE
      value: "harbor.local.lan/apps/$(params.gitRepoName):$(params.branchName)-$(params.timestamp)-$(params.imageTag)"
    - name: EXTRA_ARGS
      value:
        - "--insecure-registry=harbor.local.lan"
        - "$(tasks.fetch-secrets.results.build-args[*])" # We are passing the --build-args here

This actually worked. The build succeeded, the secrets were available during npm run build, and the image was pushed to Harbor.

The problem? Security. Secrets passed via CLI arguments in a Pod spec are completely visible when you run kubectl get pod -o yaml. They show up in logs, in manifest history, in audit trails. Anyone with cluster access could inspect the PipelineRun and see the raw credentials. Not ideal when you’re trying to build a secure pipeline.

If you’re in an environment where this visibility isn’t a concern (maybe you’re the only person with cluster access, or the secrets aren’t particularly sensitive), this approach might work for you. For me, it was a non-starter.

Attempt #2: Vault agent sidecar injection

The second approach used HashiCorp’s Vault Kubernetes injector. This tool is genuinely clever: you add annotations to your Pod spec, and Vault automatically injects a sidecar container that fetches secrets and mounts them as files in a shared volume. The application just reads from /vault/secrets/ and everything works.

To ensure credentials are never visible to the naked eye, Docker secrets is the way to go. This required switching from Kaniko to Buildah (Kaniko’s been unmaintained for almost a year at this point, which is a separate problem). Buildah supports Docker’s native secrets mounting feature, which Kaniko doesn’t.

Here’s the updated pipeline:

# Part of a Tekton Pipeline
- name: build-and-push
  runAfter: ["fetch-source"]
  taskRef:
    resolver: hub
    params:
      - { name: name, value: buildah }
      - { name: kind, value: task }
      - { name: version, value: "0.9.0" }
  params:
    - name: IMAGE
      value: "harbor.local.lan/apps/$(params.gitRepoName):$(params.branchName)-$(params.timestamp)-$(params.imageTag)"
    - name: TLSVERIFY
      value: "false"
    - name: BUILD_EXTRA_ARGS
      value: "--secret=id=build-env,src=/vault/secrets/build-env"

And the trigger template with Vault annotations:

# Part of a Tekton TriggerTemplate
taskRunSpecs:
  - pipelineTaskName: build-and-push
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: "tekton-pipelines"
        vault.hashicorp.com/agent-inject-secret-build-env: "homelab/$(tt.params.git-repo-name)/$(tt.params.git-branch-name)"
        vault.hashicorp.com/agent-inject-template-build-env: |
          {{- with secret "homelab/$(tt.params.git-repo-name)/$(tt.params.git-branch-name)" -}}
          {{- range $k, $v := .Data.data }}
          export {{ $k }}={{ $v }}
          {{- end }}
          {{- end -}}

This worked brilliantly for projects that had secrets. The sidecar would inject the file, Buildah would mount it, the build would succeed. Clean, automated, no secrets in logs.

The problem was almost no repositories in my homelab need build secrets. For those projects, the Vault path homelab/example-site/main simply doesn’t exist in Vault.

When the Vault injector can’t find a secret, it fails. The entire Pod fails. The build fails. The pipeline fails. There’s no graceful fallback or “continue if missing” option.

I spent longer than I’d like to admit trying to make optional secrets work with the injector before accepting that I was fighting the tool. The Vault sidecar is designed for deployments where you know secrets exist and you need them at runtime. CI pipelines are different. Some builds need secrets, some don’t, and you want both to succeed without manual intervention.

If you’re in a situation where all your builds always need secrets, and those secrets always exist in Vault, the sidecar approach is genuinely excellent. It’s automatic, secure, and requires minimal configuration. For my mixed-use homelab, it wasn’t flexible enough.

## The final winner: explicit fetching

After two attempts that worked but had dealbreaker limitations, I went back to basics and simplified approach. I created a dedicated Tekton task that explicitly fetches secrets immediately before the build step.

The architecture is almost boring:

1. A Tekton task authenticates to Vault using the Pod’s Kubernetes ServiceAccount
2. Fetches the secrets for this specific repository and branch (if they exist)
3. Transforms the JSON into shell-sourceable format (export KEY=VALUE)
4. Writes to a file in the shared workspace
5. The build step sources this file, loading variables into memory
6. Build completes, workspace is destroyed, file disappears

Here’s the complete task:

# Part of a Tekton Pipeline
- name: fetch-build-secrets
  runAfter: ["fetch-source"]
  workspaces:
    - name: source
  params:
    - name: SECRET_PATH
      value: "homelab/$(params.gitRepoName)/$(params.branchName)"
  taskSpec:
    params:
      - name: SECRET_PATH
    workspaces:
      - name: source
    steps:
      - name: fetch-secrets
        image: hashicorp/vault:latest
        script: |
          #!/bin/sh
          set -e

          apk add --no-cache jq

          export VAULT_ADDR=http://vault.vault.svc:8200
          SECRET_PATH="homelab/$(params.gitRepoName)/$(params.branchName)"
          OUTPUT_FILE="$(workspaces.source.path)/.build-secrets"

          # Log in to Vault
          LOGIN_JSON=$(vault write -format=json auth/kubernetes/login \
            role=tekton-pipelines \
            jwt=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token))

          export VAULT_TOKEN=$(echo $LOGIN_JSON | jq -r .auth.client_token)

          # Fetch secrets and format as shell exports
          vault kv get -format=json "$SECRET_PATH" 2>/dev/null | \
            jq -r '.data.data | to_entries[] | "export \(.key)=\(.value)"' \
            > "$OUTPUT_FILE"

          if [ -s "$OUTPUT_FILE" ]; then
            echo "Successfully retrieved secrets from Vault. Wrote $(wc -l < $OUTPUT_FILE) vars to workspace."
          else
            echo "No secrets found at $SECRET_PATH. Creating empty file."
            touch "$OUTPUT_FILE"
          fi

The critical piece is 2>/dev/null combined with the file existence check. If the secret path doesn’t exist in Vault, the vault kv get command fails silently, and we create an empty file instead. The build continues without secrets. No errors, no manual intervention.

The Buildah build step then references this file:

# Part of a Tekton Pipeline
- name: build-and-push
  runAfter: ["fetch-build-secrets"]
  taskRef:
    resolver: hub
    params:
      - { name: name, value: buildah }
      - { name: kind, value: task }
      - { name: version, value: "0.9.0" }
  params:
    - name: IMAGE
      value: "harbor.local.lan/apps/$(params.gitRepoName):$(params.branchName)-$(params.timestamp)-$(params.imageTag)"
    - name: TLSVERIFY
      value: "false"
    - name: BUILD_EXTRA_ARGS
      value: "--secret=id=build-env,src=$(workspaces.source.path)/.build-secrets"

And the Dockerfile uses Docker’s secrets mounting:

RUN --mount=type=secret,id=build-env \
    . /run/secrets/build-env && \
    npm run build

The --mount=type=secret creates a temporary in-memory filesystem at /run/secrets/build-env that only exists during this specific RUN instruction. The secrets are sourced (loaded into the shell environment), the build executes with those variables available, and then the mount disappears. The final image contains zero trace of the secrets. No environment variables baked in, no files in the filesystem, nothing in the layer history.

And that’s it! We have secrets at build-time, hidden from malevolous logs, and not all my apps require secrets to be present.

The world shines once again.