The model is the core. The harness is everything else.

The five layers of a production harness

If the harness matters as much as the model, the next question is what's in it. Walking it from the bottom up shows why each layer exists and what breaks if you try to go without it. Pick a layer:

Context isn't missing. It's everywhere.

The hard part of giving agents the right context isn't a shortage of it. Enterprises have enormous amounts. The hard part is that it lives everywhere: in unstructured documents in SharePoint and wikis, in structured tables in OneLake and data warehouses, in productivity apps like Outlook, Teams and Word. No single retrieval method reaches all of it.

Where classic RAG runs out

The standard answer of the last two years, classic retrieval-augmented generation, was never designed for this. It is a one-shot pattern: embed the user's question, search a single index, return the top-k results, pass them to the model. It works for simple questions against a small, clean corpus. It breaks when the question is ambiguous, the corpus is heterogeneous, the answer requires combining sources or the first retrieval comes back empty. The agent has no way to recover from a bad retrieval.

When RAG isn't working well, the whole agent isn't working well. One-shot is the wrong shape for the problem. — Marco Casalaina's point, paraphrased

The fix is to treat retrieval as something the system can iterate on, the same way an agent iterates on a task: plan the query, try a source, evaluate the result, try a different source if the first came back empty, and combine what you find. See it for yourself:

The context layer as a service: the four IQs

Microsoft's answer is to ship the context layer itself as a set of services. There are four of them, collectively called Microsoft IQ. Each is a headless service that agents call through MCP, and all of them share one architectural idea: retrieval-as-a-subagent.

The same loop, applied to tools

So far this is all about data, but the same loop applies to tools. An agent with a few tools can list them all in its prompt. One with dozens can't: listing them burns context on every call and slows the model down as it scans the list. The fix mirrors agentic retrieval. Instead of exposing every tool, the agent searches for the right one, gets it back and calls it.

Foundry exposes this as tool search, and it's the pattern OpenAI's and Anthropic's agents have converged on too. The loop that pulls in knowledge also pulls in capability: the agent looks up what it needs at the moment it needs it, instead of carrying everything up front.

An identity and a place to act

Retrieval gets the agent the right information, but information alone doesn't finish a task. An agent that knows the customer is unhappy still has to write the apology email and refund the order. To do that responsibly it needs two things:

Guardrails at the tool boundary

Identity bounds what an agent can reach. Guardrails bound what flows through it once it acts. A chatbot only had to screen the user's prompt and the model's reply. An agent has more surface to defend, because it also reads tool outputs and retrieved documents, and any of those can carry an instruction the user never typed. That's how indirect prompt injection works. Try it:

The fix is to move guardrails to the tool boundary, screening tool inputs and tool outputs rather than just the model's input and output. Foundry runs its own classifiers at the tool-call and tool-response level, on top of whatever safety the model already has. And because these controls live on a shared tool layer rather than inside each agent, a team configures them once and every agent that uses those tools inherits them, instead of re-implementing the same guardrails, credentials and policies agent by agent.

The other half: evaluating agents in production

Context is half of what determines whether an agent survives production. Evaluation is the other half. An agent with the right context can still drift, regress or fail in new ways as traffic patterns change. Evaluation closes the loop: it tells you when something has changed, and whether the agent is doing what it's supposed to do.

Continuous evaluation, not a pre-ship checkbox

Most teams treat evaluation as a pre-ship checkbox: run a test suite once, see green, ship. That works for traditional software because traditional software is deterministic. An agent is not. The same prompt against the same model can produce different responses, the model itself changes when the provider ships an update, and the data it retrieves changes daily. A pre-ship suite catches a snapshot of behavior, not a guarantee of it.

Continuous evaluation closes that gap. It runs against live traffic, samples real user interactions, scores them on whatever criteria the team has defined and surfaces results into the same observability pipeline that handles infrastructure incidents. When quality regresses, the team finds out from the same alerting system that pages them when a service goes down. The same evaluations can also run earlier, as a gate in the deployment pipeline.

Rubrics: does it work, or does it work right?

Most evaluation systems rely on generic metrics like groundedness, coherence and task completion. They're useful, but they have a ceiling. As Casalaina put it, generic metrics tell you whether the agent works; they don't tell you whether it works right. Consider a restaurant-booking agent:

A rubric is a specific, use-case-tied check that asks a yes-or-no question about something the agent should be doing. The eval system runs the agent against a set of test interactions, scores each rubric per interaction and rolls up the results. The team sees not just whether the agent works but which specific behaviors it gets right and which it doesn't. Rubrics beat generic metrics because the team that owns the agent writes them, so they reflect what the agent is actually supposed to do.

You don't have to start from a blank page. Foundry can draft a rubric by reading the agent's configuration and its production traces, proposing the dimensions worth scoring from how the agent actually behaves. The team still owns and edits the result, but it starts from real traffic.

The self-improving loop

Microsoft built rubric-based evaluation into its Agent Optimizer suite, which also makes rubrics actionable: when one fails, it improves the agent automatically. It can rewrite the system prompt to make the missing behavior explicit, adjust how the agent uses its tools, change which sources it prioritizes, swap the underlying model and tune the agent's skills. Rather than testing one fix at a time, it produces several candidates in parallel, scores each against the rubrics and promotes the best one as a new agent version.

Evaluation isn't a phase that ends. It's a layer that runs alongside the agent, with rubrics that evolve as its job evolves and an optimizer that turns regressions into actionable changes instead of manual fixes. The team's job stops being to write the perfect agent on day one and becomes keeping the agent honest as the world around it changes.

The key lesson, whether or not you use Foundry

The thread running through the whole conversation is that the harness matters as much as the model. Hard context, hard evaluation, prototypes that don't survive: they all trace back to the same root. The work most teams treat as "around the model" is what actually determines whether the agent works. Treating the harness as something you add later is the most common reason an enterprise agent never makes it to production.

According to the source, the first two ideas are buildable on top of any platform.