A customer asks to move a service appointment. The conversation sounds straightforward. Behind it, the system must establish who is asking, find the right booking, check availability, obtain confirmation and update the calendar. If the connection drops during the update, someone needs to know whether the appointment actually changed.

That is the architecture problem. Choosing a model is one decision within it.

For an organisation already using Microsoft infrastructure, Azure offers building blocks for the surrounding service: identity, application hosting, search, integration and monitoring. The useful question is how those parts should work together for a particular business process.

The appointment journey below is an illustrative design, not a description of a completed RZK customer deployment. Recommendations should be adapted to the organisation’s systems, risk and operating requirements.

Begin with the work the agent may perform

Write down the permitted outcome before selecting services. An appointment assistant might answer general questions without authentication, retrieve a customer’s booking after verification, and change it only after an explicit confirmation. Cancelling a contract could remain outside its scope.

This distinction gives the architecture three different paths: public information, authorised retrieval and controlled changes. Each needs its own rules.

Choose the simplest orchestration that meets the need. Microsoft’s guidance distinguishes direct model calls, a single agent with tools and multi-agent orchestration. Every additional agent introduces coordination, latency and failure considerations. A bounded booking workflow may need one agent and a small set of APIs. [1]

For this example, define tools such as “get available appointments” and “reschedule this booking”. Keep the business rules in the application and booking service, where they can be tested independently of the conversation.

Give each layer a clear responsibility

Microsoft’s baseline Foundry chat architecture is a useful reference for an enterprise deployment. It brings together the user application, agent runtime, model, data services and network controls. Treat it as a starting point for design decisions rather than a compulsory shopping list. [2]

Responsibilities in the illustrative appointment architecture
LayerResponsibility in the appointment example
Conversation channelReceive speech or text, display or speak the response, offer a handover.
Application and orchestrationMaintain session context, apply workflow rules, invoke approved tools.
Identity and authorisationEstablish the caller’s permitted scope and authorise each sensitive operation.
Knowledge and business APIsReturn relevant information and perform validated booking changes.
OperationsRecord outcomes, detect failures and support investigation and recovery.

Select hosting around the runtime, networking and operational needs. A managed agent service can reduce infrastructure work; application-owned orchestration can offer more direct control. Either approach still needs a clearly defined owner for the complete user journey.

Separate the user’s authority from the application’s identity

An application needs credentials to call other services. The person using it has a separate entitlement to particular records and actions.

Managed identities allow supported Azure workloads to obtain Microsoft Entra tokens without storing application credentials. They still need explicitly granted permissions on downstream resources. [3]

A workload identity that can read a booking database does not establish that this caller may read every booking. Carry the verified user and tenant context into the application’s access checks. For telephone interactions, define an appropriate verification process; a phone number or a stated name alone may be insufficient.

A useful design test is to ask what happens when an authenticated customer requests someone else’s booking reference. The service should deny the request before sensitive details reach the model.

Retrieve knowledge within the same access boundaries

Retrieval-augmented generation, or RAG, supplies relevant content to the model at request time. It does not make every answer correct or establish permission to use the content.

Azure AI Search supports application-managed security filters and several native document-permission approaches. Its documentation currently marks native ACL/RBAC, Purview label and SharePoint ACL approaches as preview. Confirm support and production suitability for the chosen connector, API and SDK. [4]

If the application supplies filters, construct them from trusted identity context on the server. Test cross-tenant requests, group changes, removed permissions and stale indexed content. Keep source identifiers so a reader can inspect the material behind an answer.

For the appointment assistant, use approved knowledge for opening hours and preparation instructions. Retrieve live availability through the booking API. Putting yesterday’s calendar export in a search index would not make it an authoritative booking system.

Place business actions behind a controlled API

Azure API Management can provide a gateway for APIs, including services outside Azure. Capabilities depend on the selected tier and configuration. [5]

Use a gateway where it provides a useful shared control point. Keep business authorisation and validation in the service that owns the transaction. A valid token alone does not prove that a proposed appointment change is allowed.

For example, the booking service should validate ownership, permitted dates and current availability. It should accept a narrowly defined request and return a structured outcome. The agent can explain that outcome to the customer.

External content also needs boundaries. Prompt Shields can detect certain prompt and document attacks, but they do not establish factual accuracy or business authorisation. [6] Treat retrieved documents and tool output as data. A sentence inside a document must never grant a new permission.

Design the network and data location separately

Private endpoints provide private IP connectivity to supported Azure services. They do not replace identity checks. Plan DNS, routing and service-specific public-access settings together, and verify the resulting connections from the deployed application. [7]

Data location is a separate decision. For supported Foundry model deployments, Microsoft distinguishes Global, Data Zone and geography-based processing. A resource’s location alone does not establish where inference runs. Check the exact model and deployment type against the organisation’s requirements. [8]

Review the complete data path: prompts, retrieved passages, conversation state, recordings, traces and backups. Microsoft’s privacy documentation describes feature-specific storage and processing, including persistent state and abuse-monitoring arrangements. Avoid a blanket promise that an Azure agent stores nothing. [9]

Keep a written record of what each service receives, where processing and storage may occur, who can access the data and how retention is enforced. Include external CRM and telephony providers in that record.

Make voice a deliberate channel decision

Voice introduces interruption, silence, recognition uncertainty and disconnected calls. A typed conversation may tolerate a pause that feels unacceptable over the phone.

Azure’s Voice Live API offers a managed interface combining speech recognition, generative AI and speech output. It is one implementation option; validate supported languages, regions, model choices and integration requirements for the intended deployment. [10]

Test the whole conversation in English and German, including names, addresses, dates and background noise. Ask the caller to confirm important details before committing a change. Define what happens when a caller interrupts, hangs up or asks for a person.

A handover should carry an appropriate summary and the confirmed transaction status. It should also have a practical fallback when the human queue is unavailable.

Resolve uncertain outcomes before retrying

A timeout does not prove that a write failed. The booking system may have committed the appointment change while its response was lost.

Azure’s Retry pattern highlights the need to consider idempotency and the effect of repeating an operation. [11] Use bounded retries for suitable transient failures. For writes, design an operation identifier or another duplicate-prevention mechanism supported by the receiving service.

In the appointment example, record the attempted change and check its status before submitting it again. If the outcome cannot be established, tell the customer that confirmation is pending and route the case for reconciliation. The model should not improvise a successful booking message.

Define recovery targets across the dependencies. A second application instance will not restore service if the only booking API is down. Microsoft’s AI design principles recommend analysing failure modes and balancing reliability across the components of a workload. [12]

Observe completion, quality and cost

A responsive model endpoint can sit inside a failing business process. Monitor whether bookings were correctly completed, whether users needed repeated clarification and whether handovers reached a person.

Foundry observability combines evaluation, monitoring and tracing, with Azure Monitor Application Insights integration. Use the supported instrumentation for the selected runtime, and verify what it captures. [13]

Give each interaction a correlation identifier. Record tool results and relevant configuration versions, while minimising sensitive payloads. Operational traces are evidence of recorded execution; they are not a complete account of the model’s internal reasoning.

Agree on acceptance thresholds before launch. Include permission-denial tests, realistic language samples, dependency failures and duplicate-action scenarios. Review both automated scores and a representative human-reviewed sample.

Track the complete cost of a resolved interaction: model usage, speech, application hosting, retrieval, networking, telemetry and human follow-up. Azure Cost Management budgets provide alerts; they do not automatically stop consumption. [14] If spending must be bounded, implement and test application limits and a useful fallback.

What should be ready before launch?

The architecture should leave the delivery and operations teams with a practical set of decisions:

Architecture decisions before launch
DecisionEvidence to retain
What may the agent do?Approved workflow, tool contracts and escalation boundaries.
Whose data may it use?Identity design, permission tests and retrieval restrictions.
What happens when something fails?Retry rules, transaction reconciliation and tested handover.
How will changes be released?Versioned configuration, evaluation results and rollback procedure.
Who runs the service?Named ownership, monitoring, incident process and cost controls.

The strongest Azure architecture is one the team can explain in terms of the work it enables, the boundaries it enforces and the failures it can recover from. Start with a contained workflow, prove it under realistic conditions and expand from the evidence.

Discuss your Azure AI architecture with RZK — connect the agent design with your existing identity, cloud, business systems and operational environment.

Sources

Technical sources reviewed on 21 October 2026. Service availability and preview status should be rechecked during implementation.

  1. Microsoft — AI agent orchestration patterns
  2. Microsoft — Baseline Microsoft Foundry chat reference architecture
  3. Microsoft — Managed identities for Azure resources
  4. Microsoft — Document-level access control in Azure AI Search
  5. Microsoft — Azure API Management overview
  6. Microsoft — Prompt Shields
  7. Microsoft — Private endpoints
  8. Microsoft — Deployment types for Microsoft Foundry Models
  9. Microsoft — Data, privacy, and security for Models sold by Azure
  10. Microsoft — Voice Live API overview
  11. Microsoft — Retry pattern
  12. Microsoft — Design principles for AI workloads on Azure
  13. Microsoft — Observability in generative AI
  14. Microsoft — Create and manage budgets