Evaluating and Observing Azure AI Foundry Agents Before Production

Generative agents introduce a form of behaviour that traditional software testing was never designed to assure. An agent can retrieve context, reason across multiple steps, call tools, and generate free text, which means its output varies with each request rather than following a fixed path. For organisations in government, aged care, and enterprise settings, this variability creates a genuine assurance problem, because a single reassuring demonstration says little about how the agent behaves across thousands of real interactions.

Manual spot checks remain common in early experimentation, yet they cannot demonstrate consistent quality or safety at the scale production demands. A responsible approach treats evaluation, tracing, and continuous monitoring as engineering disciplines that produce evidence, not impressions. Azure AI Foundry provides native capabilities for each of these disciplines, allowing teams to measure agent behaviour systematically before and after deployment.

Why manual spot checks are insufficient

The core weakness of spot checking is that it samples a tiny, unrepresentative fraction of the input space. When a reviewer types a handful of questions and reads the answers, they observe the agent under favourable and predictable conditions rather than under the messy, adversarial, and edge case inputs that real users generate. This gives a false sense of confidence, particularly for risks such as ungrounded responses, where the agent asserts information that is not supported by its retrieved sources.

Assurance also needs to be repeatable and comparable over time, which manual review cannot provide. Every change to a prompt, a model version, a retrieval index, or a tool definition can alter agent behaviour in ways that are difficult to predict. Without a structured evaluation baseline, teams have no reliable way to detect regression, and they cannot evidence to a governance board that a new release performs at least as well as the version it replaces.

Using built-in evaluations to measure quality and safety

Azure AI Foundry includes a library of evaluators that assess agent output against defined criteria, so that quality becomes measurable rather than subjective. Quality evaluators examine attributes such as groundedness, relevance, coherence, and fluency, which together indicate whether a response is faithful to its source material and genuinely useful to the person asking. Running these evaluators across a curated dataset produces scores that a team can track, compare between releases, and present as objective evidence of performance.

Safety and risk evaluation is equally important for regulated sectors, and Foundry provides evaluators that test for harmful content categories including hateful and unfair content, violence, self harm, and sexual material, alongside checks for protected material and jailbreak susceptibility. Teams can also define custom evaluators that reflect their own policy requirements, such as adherence to a departmental tone or the correct handling of sensitive personal information. Evaluating agents specifically, rather than only the underlying model, allows teams to assess how well the agent follows its instructions, resolves the user's intent, and selects the right tools during a multi step task.

Tracing agent behaviour with observability

Evaluation tells you whether an outcome was acceptable, but tracing tells you why the agent behaved as it did. Azure AI Foundry supports tracing built on OpenTelemetry, which captures the sequence of steps an agent takes, including the prompts it constructs, the documents it retrieves, the tools it invokes, and the intermediate reasoning that leads to a final response. This visibility transforms debugging from guesswork into analysis, because engineers can inspect the exact context that produced an undesirable answer.

Integrating traces with Azure Monitor and Application Insights connects agent behaviour to the wider observability practices that enterprise operations teams already run. Correlating traces with latency, token consumption, and error rates helps teams understand not only whether an agent is safe and accurate but also whether it is efficient and reliable enough for sustained use. This combined view supports root cause investigation when an incident occurs, which is essential for meeting the accountability expectations placed on public sector and enterprise systems.

Continuous monitoring in production

Assurance does not end at deployment, because agent behaviour can drift as user patterns change, as underlying models are updated, and as retrieved content evolves. Continuous monitoring applies the same evaluators used before release to samples of live traffic, so that quality and safety metrics remain visible in production rather than fading once the launch is over. Configuring alerts against these metrics allows teams to respond quickly when groundedness declines or when harmful content indicators rise above an agreed threshold.

A disciplined monitoring practice also feeds improvement back into the development cycle by identifying real world failure cases that were absent from the original test dataset. Capturing these cases, adding them to the evaluation set, and re running the suite closes the loop between production experience and pre production assurance. Over time this builds a growing, sector specific evidence base that demonstrates the agent is being actively governed rather than left unattended after launch.

Building an evidence base for responsible AI assurance

For Western Australian government and enterprise clients, the practical value of these capabilities lies in the assurance record they generate. Evaluation results, traces, and monitoring dashboards provide the documented evidence that accountability frameworks expect, aligning with Australia's AI Ethics Principles and with broader expectations around transparency, contestability, and human oversight. This evidence allows a project team to answer, with data, the questions a risk committee will ask before approving a generative solution.

Embedding evaluation and observability early also reduces cost and delay, because problems surface during development rather than after a public facing incident. Treating assurance as a continuous engineering activity, supported by native Azure AI Foundry tooling, gives organisations a defensible path from experiment to production. The outcome is a generative agent that stakeholders can trust because its quality and safety are demonstrated repeatedly, not merely asserted once.

References

• Evaluation of generative AI applications in Azure AI Foundry
• Observability in generative AI with Azure AI Foundry
• Australia's Artificial Intelligence Ethics Principles