Nov 12, Paris: Agents in production, beyond the demo. An evening at Alan with Qovery and AWS

Kubernetes can't tell which AI agent took action: how to get per-agent attribution and audit trails in 2026

Kubernetes audit events record the credential that made the API call, never the AI agent behind it. Here is the exact audit schema, what it is missing, and five ways to get per-agent identity, scoped permissions and a reviewable audit trail with tools that exist today.

Morgan Perry
Co-founder
NOV 10, 2026 · 18 MIN
Kubernetes can't tell which AI agent took action: how to get per-agent attribution and audit trails in 2026

Key points:

  • Kubernetes audit events record the authenticated identity, not the actor. You get user.username, user.uid, user.groups, impersonatedUser, sourceIPs and a client-supplied userAgent. There is no field for the agent, the model, the prompt or an on-behalf-of chain, so ten agents sharing one ServiceAccount produce one indistinguishable identity.
  • The fix available today is identity separation. One short-lived credential per agent (a ServiceAccount token from the TokenRequest API, or an OIDC identity), bound to a namespaced Role, turns your existing audit log into per-agent attribution with no extra tooling.
  • Audit entries are not an audit trail. A reviewable trail needs five things: agent identity, accountable human, the task, the concrete diff and the approval or policy that allowed it. Raw Kubernetes gives you two of the five.
  • GitOps covers declarative changes only. Argo CD and Flux apply under their own controller identity, and Git never sees kubectl exec, port-forward, log reads, Secret reads or manual scaling, which is where agent risk actually lives.
  • With Qovery, agents call an API instead of holding a kubeconfig, on your own AWS, GCP, Azure or Scaleway account or your existing cluster. The MCP Server is read-only by default, and API Policy Tokens record each agent in the audit log as policy:<token-id>:<token-name>.

Qovery · Infrastructure control plane
Build with Claude Code, Deploy with Qovery
Learn more

Your cluster knows a credential made the change. It has no idea an AI agent did. Open any audit.k8s.io/v1 event and you will find a username, some groups, an IP and a free-text user agent string, and that is the whole identity story.

Since coding agents started touching clusters, the question I get most from platform teams is "who actually did this?" The CNCF Annual Cloud Native Survey (published January 2026) found 82% of container users run Kubernetes in production, and 66% of organizations hosting generative AI models use it for some or all of their inference. Agents are now operating that infrastructure. Here is what the audit log really records, and how to get attribution you can defend in a post-mortem.

What exactly does a Kubernetes audit event record when an AI agent makes a change?

A Kubernetes audit event records the authenticated credential, the verb, the object reference, the authorization decision and a client-supplied userAgent string, and nothing else about who or what initiated the request. If an agent uses a ServiceAccount token or a borrowed kubeconfig, the log shows that identity, full stop.

Advanced auditing reached GA in Kubernetes 1.12 (September 2018), when the audit.k8s.io group moved to v1. The Event schema has barely changed since. Here is a realistic patch from a shared bot account. Tell me which of your five agents sent it:

JSON
{
  "apiVersion": "audit.k8s.io/v1", "level": "RequestResponse", "verb": "patch",
  "user": { "username": "system:serviceaccount:default:ci-bot",
            "groups": ["system:serviceaccounts", "system:authenticated"] },
  "sourceIPs": ["10.0.4.17"],
  "userAgent": "kubectl/v1.34.1 (linux/amd64) kubernetes/abc1234",
  "objectRef": { "apiGroup": "apps", "resource": "deployments",
                 "namespace": "payments", "name": "api" },
  "requestObject": { "spec": { "replicas": 0 } },
  "requestReceivedTimestamp": "2026-11-04T03:12:09.114Z",
  "annotations": { "authorization.k8s.io/decision": "allow",
    "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"ci-bot-admin\"" }
}

You can't. The audit levels are None, Metadata, Request and RequestResponse. Metadata gives you who, when, verb and resource. To see what the agent submitted you need Request at minimum (it adds requestObject), and RequestResponse to see the resulting object. The schema itself warns that userAgent "is provided by the client, and must not be trusted", and that all but the last entry in sourceIPs can be set by the client.

What you want to knowIn the audit event? Field pathAudit level requiredWhy it is not enough for agent attribution
Authenticated identityYes: user.username, user.uid, user.groups, user.extraMetadataNames the credential, so every agent sharing that credential looks identical
Impersonated identityYes, if impersonation headers were used: impersonatedUserMetadataOnly populated when you deliberately design an impersonation flow
Source IPYes: sourceIPsMetadataAgents on the same runner, laptop or NAT share an IP, and only the last entry is trustworthy
User agent stringYes: userAgentMetadataClient-supplied free text, settable with kubectl --user-agent, never proof of anything
Verb and objectYes: verb, objectRefMetadataTells you what was touched, not who decided to touch it
Submitted object diffYes: requestObject (and responseObject)Request (RequestResponse for the response)Records the change, but attributes it only to the credential
Authorization decisionYes: annotations["authorization.k8s.io/decision"] and ["authorization.k8s.io/reason"]MetadataShows the RBAC rule that allowed it, not a human approval
AI agent nameNo field existsNot available at any levelMust be encoded into the identity before the request is made
Model, prompt or taskNo field existsNot available at any levelKubernetes has no concept of intent
Human on whose behalf the agent actedNo field existsNot available at any levelNo delegation chain unless you build one with impersonation
Approval recordNo field existsNot available at any levelApprovals live in Git, tickets or a platform, outside the API server

Three failure modes show up again and again. One shared bot ServiceAccount used by every agent. An engineer's own kubeconfig handed to a coding agent, so the agent's actions carry that engineer's name. And a cluster-admin token mounted into an MCP server, which is the worst of both.

The lever people forget is the impersonate verb, which populates impersonatedUser and can carry an agent identity if you design for it. I cover how in the setup section below.

Managed control planes add a second copy of the same log with the same identity limits, and none of them is on by default in a way you can rely on. EKS sends nothing to CloudWatch Logs until you enable each log type, and CloudWatch then keeps data indefinitely unless you set retention. On GKE, Admin Activity audit logs are always on and land in the _Required bucket with a fixed 400-day retention, while Data Access logs are off by default. On AKS, kube-audit and kube-audit-admin are resource logs that are not collected until you create a diagnostic setting.

Why can't Kubernetes natively tell a human apart from an AI agent?

Kubernetes authenticates credentials, not actors. There is no first-class non-human identity type, no on-behalf-of claim carried in an API request, and no agreed convention for "an agent did this for this human", so attribution has to be encoded into the identity you issue before the request is ever made.

ServiceAccounts were designed for long-running workloads. Their tokens say what they are (system:serviceaccount:<namespace>:<name>), never who asked. Compare that with OAuth 2.0 Token Exchange, where RFC 8693 defines an act claim "to express that delegation has occurred and identify the acting party". The API server has no native notion of an actor claim.

The TokenRequest API improved a lot, but on a different axis. Bound tokens are audience-bound, time-bound (1 hour by default for Pod-mounted tokens) and object-bound, so deleting the bound Pod invalidates the token. That fixes token lifetime and theft. It does nothing for identity granularity.

Admission controllers see request.userInfo and can police it, but only if the agent identity is distinct in the first place. A policy cannot tell agent A from agent B when both present the same token.

The standards work is real but not plug-and-play for agents yet. SPIFFE issues "short lived cryptographic identity documents" called SVIDs to workloads through attestation, and its docs do not address AI agents. SIG-Auth's structured authentication configuration went GA in Kubernetes 1.34, and its CEL claimMappings can copy JWT claims into user.extra, which does show up in the audit event. That is the closest you get to a native actor field today, and you have to wire it to your own identity provider.

Meanwhile the number of identities is exploding. The 2026 Identity Security Landscape from Idira by Palo Alto Networks (CyberArk's identity business), based on 2,930 security leaders, reports that machine identities, including AI agents, outnumber human identities 109 to 1. Agents turn that from an annual audit finding into a daily operational problem.

What does a usable AI agent audit trail actually need to contain?

A usable agent audit trail needs five fields that Kubernetes never produces together: the agent identity, the accountable human, the intent the agent was given, the concrete change applied, and the policy or approval that permitted it. Raw Kubernetes supplies identity and the change; you build or buy the other three.

  • Agent identity: unique per agent instance, or per agent and human pair, for example agent-claude-code-romaric.
  • Human principal: the person who gets paged when the change breaks production at 3am.
  • Intent: the natural-language task, stored where a reviewer reads it next to the change.
  • Concrete change: manifest diff, image tag, env var, replica count or Secret reference. That needs Request or RequestResponse audit level, or a Git diff.
  • Decision record: which policy allowed it, who approved it, and whether the session was read-only or write-enabled.

Ship all of it off the node. The API server log backend rotates files with --audit-log-maxage, --audit-log-maxbackup and --audit-log-maxsize, and anyone with root on a control plane node can edit local files. Forward to a SIEM or object storage with write-once retention.

This maps cleanly to compliance language. NIST SP 800-53 Rev. 5 control AU-3 requires audit records to establish "what type of event occurred", when, where, the source, the outcome, and the "identity of any individuals, subjects, or objects/entities associated with the event". An agent hiding behind a shared token fails that last item.

Audit trail fieldRaw Kubernetes audit loggingGitOps (Argo CD / Flux)Policy engine (Kyverno / Gatekeeper)Brokered platform API (e.g. Qovery)How you would add it yourself
Agent identityOnly if each agent has its own credentialCommit author, if the agent commits under its own nameCan require it, cannot create itPer-token or per-user identity in the platform audit logOne ServiceAccount or OIDC identity per agent
Accountable humanOnly via impersonation designPR approver and commit authorCan require an annotation naming the humanOAuth sessions act as the signed-in userNaming convention plus impersonation
Intent or taskNoPR description and commit messageCan require a task ID annotationNot stored natively, keep it in the PR or ticketTask ID annotation linked to your tracker
Concrete diffYes, at Request or RequestResponse levelYes, the Git diffSees the object at admission, does not store itChange summary plus the post-event JSON objectRequestResponse level on write verbs
Approval or policy decisionRBAC decision annotation onlyPR approvalAdmission allow or deny, plus policy reportsToken role or policy evaluated per requestApprover annotation enforced at admission

How do I get per-agent attribution on a cluster I already run?

You can get per-agent attribution in about a day with no upstream Kubernetes change: issue one identity per agent, keep tokens short-lived through the TokenRequest API, scope them with a namespaced Role, and log write verbs at RequestResponse level. Seven steps.

Step 1: one identity per agent. Use the convention agent-<tool>-<human> and never a shared bot account. I put agent ServiceAccounts in a dedicated agents namespace, so every agent automatically belongs to the group system:serviceaccounts:agents, which makes audit policy and alerting trivial.

Step 2 and 3: short-lived token, least-privilege binding. Start read-only with the built-in view ClusterRole bound per namespace through a RoleBinding, and add write verbs deliberately. Never a ClusterRoleBinding to cluster-admin.

BASH
kubectl create namespace agents
kubectl create serviceaccount agent-claude-code-romaric -n agents
kubectl create rolebinding agent-claude-code-romaric-view -n payments \
  --clusterrole=view --serviceaccount=agents:agent-claude-code-romaric
# TokenRequest API: expires in 1 hour, no Secret-based long-lived token
kubectl create token agent-claude-code-romaric -n agents --duration=1h

If the token is presented to anything other than the API server, set --audience (the TokenRequest audiences field) explicitly, and keep expirationSeconds small so revocation is fast.

Step 4: an audit policy tuned for agents. Rules match in order, so put the Secret rule first or you will write Secret bodies into your log:

YAML
apiVersion: audit.k8s.io/v1
kind: Policy
omitStages: ["RequestReceived"]
rules:
- level: Metadata                      # never log Secret payloads
  resources: [{ group: "", resources: ["secrets"] }]
- level: RequestResponse               # full diff for agent writes
  userGroups: ["system:serviceaccounts:agents"]
  verbs: ["create", "update", "patch", "delete", "deletecollection"]
- level: Metadata                      # reads, exec, logs: who and what
  userGroups: ["system:serviceaccounts:agents"]

Append these rules to your existing policy. RequestResponse is expensive on busy namespaces, which is exactly why I scope it to the agent group. Microsoft's AKS docs make the same point and recommend kube-audit-admin, which drops get and list, to control cost.

Step 5: impersonation for on-behalf-of. Flip the usual direction. Let the human (or a broker acting as the human) impersonate the agent: kubectl --as=system:serviceaccount:agents:agent-claude-code-romaric. The event then records the human in user and the agent in impersonatedUser, and authorization runs against the agent's narrow RBAC. Grant impersonate on serviceaccounts with resourceNames limited to that human's agents. Constrained impersonation, beta and on by default since 1.36, lets you also limit which verbs are allowed while impersonating.

Step 6: provenance at admission. Kyverno exposes the caller as {{request.userInfo.username}} and can match subjects; Gatekeeper exposes input.review.userInfo (username, uid, groups, extra). Reject mutations from system:serviceaccounts:agents that lack annotations such as agents.example.com/task-id and agents.example.com/approver. Note that Gatekeeper's userInfo is not available in audit mode, only at admission.

Step 7: alert on the envelope. Forward events off-cluster and alert when an agent identity gets a forbid decision, touches a new namespace, or works outside expected hours.

All of this collapses the first time someone hands an agent a cluster-admin kubeconfig to unblock a demo. The control that matters most is never issuing that credential.

Let agents ship without handing them your cluster.
With Qovery, AI agents deploy through a scoped, attributed API on your own AWS, GCP, Azure or Scaleway account, or your existing Kubernetes cluster. Read-only by default. Start in under 10 minutes.

Is GitOps enough to get AI agent attribution?

GitOps gives you a reviewable diff and a human approval gate for free, but it attributes the commit author rather than the agent, and the cluster audit log shows the controller that applied the change. It also covers only declarative changes, leaving the riskiest agent actions outside the trail.

What it does well is real. Immutable history, a pull request as the approval gate, and reconciliation events as the apply record. Argo CD's security docs say it emits Kubernetes Events "indicating the responsible actor when applicable", for example admin initiated sync to HEAD from argocd-server, while the actual apply comes from argocd-application-controller. Flux applies with its controllers' own ServiceAccounts unless you configure tenant impersonation with serviceAccountName.

What it misses entirely: kubectl exec, port-forward, log reads, Secret reads, manual scaling, and anything done in the middle of an incident. Those never touch Git.

To tag agent authorship in Git, commit trailers are the practical tool. Claude Code, for one, adds a Co-Authored-By trailer to its commits by default, and GitHub renders co-authors on the commit. Signed commits from an agent-specific key go further. Reconstructing the full story still means joining Git history with audit events on timestamp and object, which nobody does reliably under pressure.

The hybrid most teams land on: GitOps for desired state, a brokered API for everything imperative, and no raw kubeconfig for agents at all.

Which approach should I pick: raw RBAC, GitOps, a policy engine, an access proxy, or an internal developer platform?

There is no single winner, and the honest answer depends on whether you have a platform team. Raw RBAC plus a tuned audit policy delivers attribution with the most operational work, GitOps delivers approval without imperative coverage, policy engines constrain without attributing, Teleport records interactive sessions, and an internal developer platform puts identity, scope and audit behind one API the agent calls.

I will say it out loud: Argo CD plus Kyverno plus a tuned audit policy is a legitimate, complete answer if you have a platform team to run it. Each alternative also wins somewhere. Teleport proxies requests with impersonation headers tied to a named user and records kubectl sessions, which beats everything else for exec and shell access, and its Machine & Workload Identity covers bots. Backstage and Port win on catalog and self-service UX, Humanitec on platform orchestration, SPIFFE/SPIRE on workload identity, and CloudTrail, Cloud Audit Logs and Azure Monitor own the cloud-resource half of the story.

The anti-pattern is an MCP server holding a raw kubeconfig. The MCP security best practices list audit trail damage as a core risk of token passthrough ("the MCP Server will be unable to identify or distinguish between MCP Clients") and recommend starting from a minimal scope set and adding scopes only when needed. A shared kubeconfig breaks both.

ApproachPer-agent identityHuman on-behalf-of recordedApproval gateCovers imperative actions (exec, scale, restart)Blast-radius scopingSetup effortWho operates it
Raw Kubernetes RBAC + audit policyYes, if you issue one ServiceAccount or OIDC identity per agentOnly with an impersonation designNoYes, every API call is loggedNamespaced RolesMedium, ongoing tuningYour platform team
GitOps (Argo CD / Flux)Commit author only; cluster sees the controllerPR approver and commit authorYes, the pull requestNoPer app or tenant ServiceAccountMediumYour platform team
Policy engine (Kyverno / OPA Gatekeeper)Enforces it, does not create itCan require an approver annotationAdmission allow or denyAdmission only (exec is an admission request, reads are not)Per policy matchMediumYour platform team
Access proxy (Teleport)Yes, named users and bot identitiesYes, the proxy userAccess requestsYes, including session recording of execTeleport roles mapped to Kubernetes groupsMedium to highYour team or Teleport Cloud
MCP server with a raw kubeconfigNo, everything is the kubeconfig identityNoNoYes, with no record of which agentWhatever the kubeconfig allows, often cluster-wideLowNobody, which is the problem
Internal developer platform (Qovery)Yes: one API Policy Token per agent, or the user's own OAuth sessionOAuth sessions act as the user; token names identify the agentRead-only default; writes need read_write=true and a policy that allows themDeploy, restart, scale, logs via API; no raw execPer environment and service (policy token), per project and environment type (custom roles)LowQovery control plane; workloads in your cloud account

Where Qovery fits, without overclaiming: agents call the Qovery API, CLI or MCP Server instead of the cluster, and a human connects your cloud account once. Qovery runs in your own AWS, GCP, Azure or Scaleway account, or on your existing Kubernetes cluster, so workloads, the cluster and its native audit logs, and the cloud bill stay with you.

The part I care most about is the API Policy Token (in Preview). It carries no role; an OPA/Rego policy is evaluated on every request, it fails closed, revocation is immediate, and actions are attributed in the Qovery audit log as policy:<token-id>:<token-name>. Each audit entry records the user or token, the tool, the target and the change, and you can export it with qovery audit-log export.

Be clear-eyed about one thing. Inside the cluster, Qovery applies changes with its own credentials, just like a GitOps controller, so per-agent attribution lives in the Qovery audit log. That is the point of a broker: one place that knows the agent, the environment and the policy.

What does a safe default look like for giving an AI agent cluster access in 2026?

The safe default is read-only, scoped to one namespace or environment, with write access granted per task and expiring, and no agent ever holding a long-lived cluster-admin credential. Most of an agent's value is in diagnosis, which needs no write access at all.

Troubleshooting is where agents pay for themselves: events, logs and manifests before any mutation. Treat write access as an explicit, revocable switch rather than a permanent property of the token. Keep a hard split between environments: free rein on preview and ephemeral environments, gated access to production. A preview environment per pull request is the natural sandbox, because a bad agent action costs a rebuilt environment instead of an incident.

Never put Secrets or cloud credentials within an agent's reach. The agent should ask the platform to do the privileged thing, not hold the key. And alert on the shape of behaviour rather than single calls: bursts of deletes, a new namespace, activity at 4am.

Drop this into your runbook:

  • One identity per agent, named agent-<tool>-<human>
  • Tokens from the TokenRequest API, 1 hour or less, no Secret-based tokens
  • RoleBinding to view first; write verbs added per namespace, never cluster-admin
  • Audit policy: RequestResponse for agent writes, Metadata for Secrets and reads
  • Admission policy requiring task ID and approver annotations on agent mutations
  • Audit events shipped off-cluster with fixed retention
  • Alerts on forbid decisions, new namespaces and off-hours activity
  • Production writes only through an approved path (PR or brokered API)

How do I set this up with Qovery from Claude Code or Cursor?

Install the Qovery Agent Skills once and your coding agent deploys and troubleshoots through the Qovery API, attributed to your user or to a named token and scoped to an environment, instead of holding a raw kubeconfig. Four commands.

BASH|1. Agent Skills, globally (Claude Code, Cursor, Codex, OpenCode, Gemini CLI and more)
curl -fsSL https://skill.qovery.com/install.sh | bash
# 2. Or for the current project only
curl -fsSL https://skill.qovery.com/install.sh | bash -s -- --project
# 3. MCP Server in Claude Code over OAuth, read-only by default
claude mcp add --transport http qovery https://mcp.qovery.com/mcp --callback-port 4242
# 4. Read/write, only when you explicitly want it
claude mcp add --transport http qovery "https://mcp.qovery.com/mcp?read_write=true" --callback-port 4242

Your agent asks you to approve the install command, and the MCP Server opens an OAuth login in your browser. Over OAuth, the MCP Server acts with your own user permissions. For the devops_copilot tool, write mode also requires write access to be enabled in the Qovery Console on top of read_write=true.

Then ask in plain language: "deploy this app to my AWS account with Qovery" or "why is the staging environment failing?". The install adds eleven skills, including qovery-onboard, qovery-deploy, qovery-troubleshoot, qovery-preview, qovery-terraform, qovery-assess (a read-only assessment) and qovery-policy-token. The skills deploy; the MCP Server operates what already exists. For a fully autonomous agent, run /qovery-policy-token read-only on staging, can deploy the api service and the skill writes the Rego policy, tests it and creates the token.

The attribution benefit is concrete. Instead of an anonymous patch from system:serviceaccount:default:ci-bot, the record names a person or a named agent token, the environment, the tool and the change. Start with the AI agent quickstart or the Claude Code integration.

Frequently asked questions

Does Kubernetes audit logging record which AI agent made a change?

No. Kubernetes audit logging records the authenticated identity in user.username, user.groups and user.extra, plus impersonatedUser when impersonation is used, but has no field for an agent, model or prompt. If several agents share one credential, Kubernetes audit logs cannot distinguish them. Give each agent its own identity to make the audit log attribute actions per agent.

How do I give each AI agent its own identity in Kubernetes?

Create one ServiceAccount per agent, for example agent-claude-code-romaric in a dedicated agents namespace, and issue short-lived tokens with kubectl create token --duration=1h through the TokenRequest API. Bind each ServiceAccount to a namespaced Role, starting read-only. An OIDC identity per agent through structured authentication configuration works too and can carry extra claims into the audit event.

Can Kyverno or OPA Gatekeeper block or detect actions from AI agents specifically?

Kyverno and OPA Gatekeeper can block agent actions only when the agent has a distinct identity. Kyverno matches on subjects and exposes {{request.userInfo.username}}, and Gatekeeper exposes input.review.userInfo at admission time. A common Kyverno or Gatekeeper rule rejects mutations from the agents group that lack task ID and approver annotations.

Is it safe to give an MCP server a kubeconfig?

Giving an MCP server a raw kubeconfig is unsafe in most setups, because every agent connected to that MCP server acts with the kubeconfig's identity and permissions, often cluster-wide. The MCP security best practices warn that this kind of shared credential makes it impossible to distinguish clients in the audit trail. Use a per-agent, read-only, short-lived credential, or route the agent through a scoped platform API.

What is the difference between a Kubernetes audit log and an AI agent audit trail?

A Kubernetes audit log records which credential called which API with which result. An AI agent audit trail adds the accountable human, the task the agent was given and the approval or policy that allowed the change. You build an AI agent audit trail by combining Kubernetes audit logs with per-agent identities, Git or ticket history and admission policy decisions.

Do EKS, GKE and AKS control plane logs show which AI agent acted?

EKS, GKE and AKS control plane logs contain the same Kubernetes audit events, so they show the credential, not the agent. EKS audit logs must be enabled per log type, GKE Admin Activity logs are always on with 400-day retention, and AKS needs a diagnostic setting for kube-audit. Managed control plane logs only attribute actions per agent if each agent uses its own identity.

Morgan Perry
About the author
Morgan Perry

Morgan co-founded Qovery and leads engineering. He writes about Kubernetes architecture, DevOps best practices, and building resilient infrastructure at scale.

Next step

Let agents ship without handing them your cluster.

With Qovery, AI agents deploy through a scoped, attributed API on your own AWS, GCP, Azure or Scaleway account, or your existing Kubernetes cluster. Read-only by default. Start in under 10 minutes.