Platform Engineering Best Practices for Agentic Pipelines

Platform engineering spent the last decade building for exactly one kind of system actor: a human being, sitting at a terminal or clicking through a portal, whose intent had to be translated into something an API could safely execute.

Agentic pipelines break CI/CD infrastructure at the merge queue. Configurations tuned for human PR cadence become the primary bottleneck once agents generate pull requests at volume, and the frequency of "green-green" failures, where two PRs each pass CI individually but break the build together, scales up proportionally with that volume. Faire's engineering team documented roughly 20 such failures caught per month, each previously costing about two hours to trace after landing in main. The platform engineering fixes that address this instability are: speculative merge queues with configurable group sizes and the merge_group trigger wired into CI workflows; test parallelization and workflow optimization run before enabling queues at agentic scale; first-class agent identities with scoped permissions so governance can block disallowed actions before they reach the wire rather than relying on the agent's own judgment; hallucination-specific security checks that verify every dependency an agent adds actually exists in its registry; and machine-readable repository contracts covering architecture patterns, dependency policies, and testing conventions, so agents can infer platform rules directly rather than making undocumented novel choices.

Why platform engineering's human-actor assumption breaks under agentic load

The mediation layer enforced correctness, safety, and auditability, compensating for the fact that humans don't naturally speak API schemas. A developer who wanted to provision a database or open a network path had to go through a form, a template, or a reviewed pull request, and each of those checkpoints quietly did governance work: it validated inputs, recorded who asked for what, and gave someone downstream a reviewable trace of intent.

AI agents remove that checkpoint by doing the translation themselves. An agent ingests natural language, reasons directly over API schemas, and executes changes, collapsing what used to be a multi-step human-to-API translation into a single intelligent step. Systems such as Devin and OpenHands already demonstrate what this looks like in production terms: agents that navigate a codebase, implement a feature, and submit a pull request without a human writing a line of the intermediate work.

The CNCF's framing of platform engineering for the agentic enterprise states this directly. A platform can no longer assume every interaction originates from a human; it has to provide interfaces designed for both humans and AI agents, and it has to give each actor a distinct identity, scoped permissions, and a clear audit trail. Those three requirements, identity, permissions, and audit trail, map to risk categories like skill authorization and runtime isolation, which are identity problems at the skill execution layer. They are the three load-bearing functions the old mediation layer used to perform invisibly, and they are the three threads this piece follows through the rest of the agentic pipeline, from the moment an agent authenticates to the moment its code lands in production.

Giving AI agents a first-class identity in the platform

Identity is the most immediate gap to close, because nothing else in an agentic pipeline can be governed correctly without it. If the log says a human did something an agent actually did, every downstream security and compliance process is reasoning about a fiction.

The CNCF post is specific about why uniform governance has to sit on top of distinct actor types rather than erase the distinction between them. A developer using a self-service portal, an SRE running a CLI or GitOps workflow, and an AI agent invoking an MCP server all interact with the platform differently, but each needs to be governed through the same consistent security controls, policy boundaries, and operational guardrails. Consistent governance across different actor types starts with each actor carrying its own identity: not a service account borrowed from a human workflow, but a purpose-built actor identity with a permission envelope scoped to the task it actually performs.

Microsoft's platform engineering guidance shows what that looks like in a running system. An identity architecture diagram sets identity boundaries for which Microsoft Graph operations a given agent may perform, and if an action would cross an undefined trust boundary, the system blocks it before it executes. That is the scoped-permission principle made concrete: an agent that provisions infrastructure has no business touching identity boundaries or reading secrets outside its task, and the platform should make that structurally true rather than relying on the agent's good judgment.

The stakes of getting this wrong are not hypothetical. The OWASP Agentic Skills Top 10, cited in the MLflow production agent guide, ranks skill authorization third and runtime isolation sixth out of ten risk categories, and both show up as identity problems at the level of individual skill execution, not at the level of the agent's overall behavior. The EU AI Act adds a regulatory dimension: Article 14 requires human oversight mechanisms for high-risk AI systems, with verified confirmation steps mandated for certain decision classes, including biometric identification under Annex III, point 1(a). A confirmation step only means something if the system can say which agent took the action being confirmed, and that is what a first-class agent identity gives you.

The objection platform teams raise here is that service accounts already solve this problem, since automation has used them for years. An agent composes novel action sequences at runtime, choosing a path through an API surface that no one wrote down in advance, so there is no fixed scope a shared service account could safely be assigned.

Why IaC remains the audit ledger when agents generate it

Once an agent has an identity, the next question is what it did and whether it was allowed to do it. Infrastructure as Code remains the answer, though its job inside an agentic pipeline is different from the job it held when humans wrote every line. Agents do not replace IaC: they change its function, from the primary way intent gets expressed into the platform's verifiable record of what happened.

Microsoft's platform engineering guidance makes this case: even in workflows where agents generate most of the code, IaC still functions as the canonical ledger against which the platform's actual state is validated and, when it drifts, remediated. Agents operate through IaC today. They do not operate around it, and the reasons IaC earned its place in regulated change management, a deterministic desired-state model, a versioned history, a reviewable plan, and a reconciliation path for drift, are exactly the properties an agentic pipeline still needs for rollback and incident response.

But the longer arc points somewhere more interesting than a static ledger. As agents gain the ability to reason directly over cloud provider schemas, apply inline policy enforcement as they act, and generate a complete, immutable trace from intent to plan to API call to validation to evidence, IaC becomes one of several compiled output formats rather than the sole source of truth. The actual input, in that world, is a layer of higher-order documents: security and compliance policy, architecture decision records, approved reference diagrams, and explicit statements of human intent.

Microsoft's own example makes the shift tangible. An approved hub-and-spoke diagram governs network segmentation, and an agent can read that diagram to determine which virtual networks, route tables, network security groups, and DNS zones it may create; it applies the change, then writes the evidence of what it did back to the architecture decision record. The authoritative record has moved upstream, from a description of how the infrastructure is built to a record of why it was allowed to exist in that form, with the agent's execution trace serving as the binding link between the two.

That has a direct consequence for how platform teams manage their own documents. Security standards, architecture decision records, and approved reference architectures become the new ledger, and they need the same version control and review rigor that HCL or YAML files received under the old model.

How runtime governance closes the identity-and-IaC gap

Identity establishes who acted. Neither one, by itself, stops a bad action from executing in the first place, which is the job runtime governance has to do.

The MLflow production agent guide treats this as a non-negotiable architectural principle: policy enforcement has to sit beneath the model layer, deterministic and separate from the agent's own reasoning, so that undesired actions get stopped before they reach the wire. The guide cites the Microsoft Agent Governance Toolkit for a related and stronger claim: a properly governed action is not merely unlikely to happen, it is structurally impossible. If a governance model depends on the agent choosing not to cross a line, it is a prompt, not a control surface.

The architecture described in ISHIR's work on agentic data pipelines gives this principle a concrete place in the stack: a dedicated governance layer that enforces policy, defines what can run autonomously versus what needs approval, and logs every decision so it stays traceable. That source's six-layer pipeline model puts governance last, positioned deliberately after the action layer rather than beside it, so that governance is the thing standing between a proposed action and its execution.

A single governance policy will not serve every kind of agent a platform runs. Infrastructure agents, code-generation agents, and incident-response agents each need their own policy set, and a monolithic policy engine trying to cover all three ends up either too permissive for the riskiest one or too restrictive for the safest. They're the minimum viable control surface for a non-human actor, functionally equivalent to what RBAC provides for human users.

How do agentic pipelines break CI/CD infrastructure and what fixes the instability?

Governance architecture allows or blocks an agent's action before it happens. The merge queue determines what happens to the resulting code once it is sitting in a pull request alongside dozens of others, and it is where agentic throughput turns into a concrete operational problem rather than an abstract governance one. The merge queue is the central throughput mechanism in these pipelines, and its configuration, tuned for the pace of human pull requests, becomes the main bottleneck once agents start generating PRs at volume.

The core job of a merge queue hasn't changed: it validates every change against the exact state of the branch it's about to join, and that property becomes more valuable, not less, as the rate of incoming changes rises and the risk of a "green-green" failure, two PRs that each pass CI individually but break the build together, compounds with volume. Faire's engineering team put a number on what this protects against: their merge queue setup catches roughly 20 green-green issues a month before they reach main, and each one used to cost about two hours to track down and fix once it had already landed. At the PR volume an agentic pipeline can generate, the frequency of these issues scales up proportionally, and so does the cost of not catching them in the queue.

GitHub's native Merge Queue illustrates what the mechanics look like in practice: it lives inside branch-protection rules, builds temporary refs for testing, and supports speculative batching with configurable minimum and maximum group sizes, though a CI workflow only runs against the queue if it explicitly adds the merge_group trigger. The main constraint teams run into is CI speed multiplied by queue depth: a 20-minute test suite with a deep queue produces a wait time that compounds across every PR behind it. Workflow optimization and test parallelization need to happen before a team turns on merge queues at agentic scale. The maximum group size setting controls how many PRs share one temporary branch, and the default of 5 reflects a real tradeoff: too small a group creates excessive temporary branches, too large a group raises the odds that the batch fails because one bad PR drags down several good ones.

The logical endpoint is a merge queue that an agent operates directly, handling conflict resolution, CI failure triage, and routine queue management on its own, so that human attention gets reserved for decisions that actually require judgment rather than queue housekeeping. That is the same composable infrastructure principle running through this entire piece, applied narrowly to the merge function, and it sets up the next problem directly: what is actually inside the code that's moving through that queue at agent speed.

Security scanning on agent-generated code requires different defaults than on human-authored code

Code generated by an agent fails in ways that scanning tools built around human authorship were never asked to catch, and the most consequential of these failure modes is hallucination: an agent inventing a dependency or an API call that does not exist. A hallucinated API call becomes a runtime failure, and a hallucinated dependency becomes a supply chain risk the moment someone, or some automated process, tries to resolve it, and a lint check or a style review built to catch human mistakes will not catch either one.

A pipeline built for agent output needs a check that simply didn't exist as a default for human code: verifying that every package an agent adds to a dependency manifest actually exists in its registry. Agents do invent plausible-sounding names, so you need this check, not just want it.

The industry has already started building this in as a default rather than a custom addition. As of June 9, 2026, GitHub turned on automatic CodeQL, Advisory Database, and Secret Scanning analysis for code produced by any external agent, including both Claude and OpenAI Codex, making this scanning available natively rather than something each team has to wire up on its own. This class of risk is now treated as a baseline platform concern rather than an edge case a handful of security-conscious teams worry about.

The best-practice shape for an agentic coding workflow follows from that premise. Run the agent in a local sandbox, require CI and PR review before anything merges, and treat security scanning as a gate at the PR stage, not as an advisory comment someone can dismiss. The MLflow production agent guide locates most of this risk precisely: it clusters at the plugin and skill execution layer, not at the level of the language model itself, and the OWASP Agentic Skills Top 10 covers skill authorization, supply-chain integrity, and runtime isolation as the categories that matter most. Securing the model's reasoning is not the same project as securing what its skills are permitted to touch, and the second project is where the production risk actually concentrates.

Making the repository the interface agents understand

Everything above, identity, the IaC ledger, runtime governance, the merge queue, and security scanning, depends on an agent being able to read the repository it's working in and correctly infer what it's allowed to do. Most repositories were written for human readers who already know the unwritten rules of the codebase, and an agent has no access to those unwritten rules unless someone writes them down in a form it can parse.

A repository operating inside an agentic pipeline needs documented architecture patterns, dependency policies, testing conventions, file organization rules, and security requirements, structured so that an agent can parse and enforce them directly rather than merely having a human read them during onboarding. An agent working against a repository with no machine-readable contract will fail in one of two directions: it will over-ask, interrupting humans for routine decisions the platform should have already settled, or it will under-ask, making novel choices on its own that no one on the platform team has visibility into until something breaks.