Architectural Decisions

The technical architecture of a Voice AI Agent project has three layers. The speech layer covers the STT, LLM and TTS chain. The integration layer connects to enterprise systems. The oversight layer is responsible for monitoring, security and audit. It matters to see that each layer holds decisions of its own:

In the speech layer the first decision is the speech recognition engine: is it optimised for a single language, and how well does it handle accents. The second is which language model runs which task; simple queries want a small, fast model, while complex reasoning needs a large, slower one. The third is where speech synthesis sits between latency and naturalness.

In the integration layer you decide how many enterprise systems connect, in what order, and whether each connection is synchronous or asynchronous. A synchronous connection waits for an immediate answer, as a balance enquiry does. An asynchronous connection can move on without waiting, as when a request is processed in the background. Failing to make that distinction early can force the whole conversation flow to be redesigned later.

In the oversight layer the depth of logging becomes clear: which data is kept and for how long, whether monitoring is real time or retrospective, and who can reach that data.

These decisions need to settle early, because each one affects the others. A requirement that data cannot leave the country, that is data sovereignty, determines the deployment model directly; the deployment model shapes the integration architecture; and the integration architecture limits the latency budget you can work within. That is why a single decision you defer with "we'll change it later" actually reopens four or five decisions at once.

Criteria to weigh when choosing a deployment model:

The three are not mutually exclusive. A pattern seen often in the field is a hybrid deployment: the speech layer, which is most sensitive to latency and to data, is kept on-premise or in a private cloud, while the analytics and oversight layer runs in the cloud. The right question is not "which one do we choose" but "which layer should run in which environment".

A common mistake: taking the architecture chosen for the pilot to be the production architecture. A pilot usually starts in the cloud with loose integrations so that it can be stood up quickly. When the project moves to production, data sovereignty, deep integration with existing systems or audit requirements appear, and the pilot architecture cannot meet them. The result is a rebuild from scratch. To avoid that risk, the outline of the production architecture should be settled before the pilot begins, particularly the deployment model and data sovereignty. The pilot should be a smaller version of that architecture rather than a shortcut entirely independent of it.

CBOT's platform architecture brings the three layers together in a single deployment model and can run in the cloud, in a private cloud, fully on-premise or in the hybrid model described above.