A caller wants to move a service appointment. The agent understands the request, finds an available slot and gives a clear answer. It is a convincing demonstration.

Production introduces the questions that the demonstration can leave unanswered. Has the caller been verified? Can they change this appointment? What happens if the calendar accepts the change but the connection fails before returning a confirmation?

These are deployment decisions. A capable model helps the conversation, but the service also needs controls around data, permissions, business actions and recovery. OWASP describes the risks of giving an agent excessive functionality, permissions or autonomy under excessive agency. [1]

For teams moving a voice or chat agent into real operations, the useful starting point is a clearly defined workflow and an access model that remains enforceable when the conversation goes wrong.

Start with a defined workflow

Write down the agent's job in terms that can be tested. “Manage customer service” leaves too much open. “Answer approved service questions, check appointment availability and request a booking change for a verified customer” gives the engineering team a boundary.

Define the allowed sources, the systems involved, the actions available and the conditions that require a person. Include what happens when the agent cannot complete its task.

For an appointment service, an initial permission map might look like this:

Illustrative appointment permission map
ActivityAccess boundaryExecution condition
Explain opening hours or service optionsApproved public knowledgeAnswer from the maintained source.
Retrieve an existing appointmentThe verified customer's relevant recordCheck authorisation before retrieval.
Change an appointmentA specific booking operationApply business rules and obtain confirmation of the proposed change.
Override cancellation charges or export customer recordsOutside the workflowRefer to an authorised member of staff.

Illustrative appointment workflow.

This also makes later expansion easier to review. Adding a new action means changing an explicit permission map, rather than discovering that a broad connector already allowed it.

Carry identity through every system

An agent should not inherit trust simply because it runs inside the company network. NIST's zero trust guidance separates access decisions from assumptions about network location or ownership. [2]

Distinguish the person requesting help from the software identity calling a business system. If a signed-in employee asks about a document, the retrieval path should preserve that employee's access rights. A powerful backend account should not quietly make restricted documents available.

In Azure, managed identities can remove the need to store service credentials for supported workloads and resources. They still need appropriately scoped role assignments. User-delegated access and application access serve different purposes; choose the pattern that matches the workflow. [3]

For a voice agent, define a separate verification process before exposing account information or accepting account changes. Verify the caller through the approved process before treating their stated name as an authenticated identity. Where verification or permission cannot be established, route the request to a person or another approved verification channel.

The same check belongs at the tool boundary. Before a call executes, the application needs to establish which user or customer is involved, which resource is being targeted and whether the requested operation is permitted.

Keep unauthorised data out of the model context

Retrieval-augmented generation, or RAG, lets an application supply relevant source material to a model. The search results still need access controls.

Apply customer, tenant and document permissions before retrieved content enters the model context. In a shared system, membership of one customer organisation does not automatically grant access to every document within it. Microsoft's secure RAG guidance recommends filtering every retrieval request using the relevant authorisation context. [4]

Apply the same separation to conversation history, cached responses and persistent agent memory. Microsoft also recommends isolating agent memory per user and tenant. [12] A useful release test is to run two customers through similar requests and check that neither can recover the other's records or conversation details.

For voice services, map the full data journey: telephony, transcription, model processing, business tools, logs and storage. Audio and transcripts can contain personal or sensitive information. Microsoft's speech-to-text documentation also distinguishes processing and storage behaviour across features, so one setting should not be treated as a guarantee for the whole voice pipeline. [5]

Decide what each component needs, where that information is processed, who can access it and when it is deleted. Keep the answer specific to the services and configuration being deployed.

Treat external content as a possible instruction attack

An agent may read a webpage, an email, a support ticket or a document that contains instructions planted by someone else. If it follows those instructions instead of the authorised task, the result can be a prompt injection attack.

The content might look routine: a note telling a support agent to export records “for verification”, or a retrieved page asking it to use a different destination for a tool call. OWASP's prompt injection guidance notes that RAG and fine-tuning do not fully remove this vulnerability. [6]

Keep retrieved content separate from trusted instructions, label its origin and test how the agent handles hostile material. Filters and model safeguards can help, but the permissions and execution controls must still hold if the model accepts the attacker's instruction.

For the appointment example, a malicious note must not create an export capability that the application never granted. This is a useful way to judge the design: test the access boundary as well as the agent's answer.

Put a check between the plan and the action

The model can propose an operation. The execution layer should independently validate its permission, target and parameters before calling the business system.

Use narrow operations such as “change this appointment to this permitted slot”. An unrestricted database query, shell command or general HTTP request gives the agent much more room to act than the task requires.

Check values against the tool's schema and business rules. A structurally valid request can still target the wrong customer's booking or an unavailable appointment. Required approvals should be bound to the actual operation and its parameters, with an expiry. For a voice booking, read back the date, local time and service location before asking the caller to confirm. If those details change, obtain fresh confirmation. OWASP's agent security guidance explicitly recommends separating decisions from execution and binding approval to the exact action. [7]

Set review requirements according to the consequences. Answering an opening-hours question and changing access permissions need different controls. Keep human review focused on sensitive, irreversible or exceptional actions, while ordinary permitted work follows defined rules.

Also design for uncertain outcomes. A timeout does not prove that an operation failed: the receiving service may have completed it before the response was lost. If repeating the request can create another booking or charge, a retry can produce duplicate effects. [8]

For a booking change, use the downstream system's supported duplicate-prevention mechanism, often called idempotency, where available, and reconcile the booking status before repeating an uncertain write. Tell the caller when confirmation is unavailable. The agent should only describe a change as complete when the workflow has reliable evidence of completion.

Contain the runtime and review every connector

Keep development and production separate, and give the runtime only the files, secrets and network destinations required for its job. Use isolation appropriate to the workload and test the boundary. A sandbox adds a layer of containment; its configuration and surrounding controls still need testing.

Anthropic's published containment work describes enforcing limits through sandboxes, virtual machines and network egress controls, alongside model safeguards. Constrain the resources an agent can reach alongside the behaviour expected of it. [9]

Treat connectors, packages and tool servers as production dependencies. Review their publisher, permissions, update process and access to data. Maintain an inventory so a connector can be disabled or updated when its risk changes.

Where Model Context Protocol (MCP) is used, review the security of the particular implementation and connection. For OAuth-protected MCP servers, the protocol's security guidance prohibits accepting tokens intended for another resource and passing them unvalidated to downstream services. A standard connection method still needs correct authentication, authorisation and token handling. [10]

Design the evidence and the handover

A transcript shows the conversation. Operations also need a record of the actions taken: the requesting identity, agent version, tool, target, policy result, approval where required and confirmed outcome. Use an interaction identifier to connect these events.

Keep that evidence proportionate. OWASP recommends excluding secrets and access tokens from logs and applying special handling to sensitive personal information. Protect logs against unauthorised access or alteration, and set retention and deletion rules. [11]

In practice, a protected reference to a customer record may be enough for an investigation; a full copy of the conversation may be unnecessary. Test the telemetry configuration before live traffic so debugging does not create a second store of sensitive data.

Set limits on execution time, tool calls, retries and spend. A managed platform does not remove responsibility for the agent's configuration, permissions or orchestration. Microsoft's agent shared responsibility guidance includes step limits, loop detection and cost ceilings among the controls that deployment teams need to consider. [12]

Make the fallback operational. Define who receives a handover, what context they receive and what happens outside staffed hours. Include the verification status, proposed change and last confirmed system result. If the booking status is uncertain, say so in the handover. This gives the specialist a useful place to resume the request.

Test before widening access

Evaluate the deployed workflow, including its real permission paths and failure behaviour. A successful conversation alone is insufficient evidence for release. NCSC's secure deployment guidance calls for appropriate security evaluation before release, alongside incident procedures and clear information about limitations. [13]

For the illustrative appointment workflow, an initial test set could include:

Proposed release acceptance tests
TestExpected system behaviour
An unverified caller requests appointment detailsNo protected details are retrieved or disclosed; the approved verification or handover path is offered.
A verified customer requests someone else's bookingResource-level access is denied.
A retrieved note instructs the agent to export recordsThe note grants no permissions; an unauthorised operation cannot execute.
A calendar write times out after submissionThe outcome is reconciled before an unsafe retry or success claim.
Two customers use similar conversation historiesStored state and retrieved results remain separated.
Repeated tool failures consume the retry budgetExecution stops and the defined fallback is used.

Use these proposed acceptance tests as a starting point. Add cases for supported languages, noisy audio, ambiguous spoken dates, integrations and the business rules in the actual service.

Begin with test data and limited access. Move to a controlled pilot with explicit acceptance criteria, then expand only the capabilities that meet them. Agree the criteria with the service owner before launch: authorised data access, correct actions, recoverable failures, usable handovers and an affordable operating envelope all matter.

Keep prompts, tool definitions, policy rules and deployment configuration under version control. Record the model version or deployment identifier where the provider exposes it. NCSC's operations guidance highlights that changes to data, models and prompts can change system behaviour and should be reflected in testing and update processes. [14]

Provide a way to pause actions, revoke access and return to a known configuration. Rolling back the application does not reverse a booking, payment or message already sent to another system. Recovery planning needs to address those business effects separately.

Before the first live request

The deployment team should be able to demonstrate:

  • An agreed workflow, with a named business owner and technical owner.
  • A working identity check and authorisation for each target resource.
  • Data and conversation separation across users and customers.
  • Narrow tools with validated inputs and appropriate approval rules.
  • Tested handling of hostile content, timeouts and duplicate requests.
  • Protected audit events, defined retention and actionable alerts.
  • Working handover, incident and service-disable procedures.
  • A reviewed release record and a practical recovery plan.

The appointment example ends well when the right customer's booking changes once, the result is confirmed and the operating team can explain what happened. If the request cannot be completed, the caller gets a clear next step.

That is the standard to design for: a useful conversation supported by access controls, observable actions and people who can take responsibility for the outcome.

Planning a voice or chat agent for your organisation? Discuss your use case with RZK, or explore AI Agent Engineering and Governance, Security & Reliability.

Sources

Primary documentation consulted on 20 October 2026. The examples, permission map and acceptance tests above are illustrative engineering recommendations.

  1. OWASP — LLM06:2025 Excessive Agency
  2. NIST — SP 800-207: Zero Trust Architecture
  3. Microsoft — Access patterns and controls for AI agents
  4. Microsoft — Design a secure multitenant RAG inferencing solution
  5. Microsoft — Data and privacy for Speech to text
  6. OWASP — LLM01:2025 Prompt Injection
  7. OWASP — AI Agent Security Cheat Sheet
  8. Microsoft — Retry pattern
  9. Anthropic — How we contain Claude across products
  10. Model Context Protocol — Security Best Practices
  11. OWASP — Logging Cheat Sheet
  12. Microsoft — AI agent shared responsibility model
  13. NCSC — Secure deployment
  14. NCSC — Secure operation and maintenance