TOTVS presents data-layer architecture for AI agents using data mesh, low-latency databases, and semantic models

The presentation highlighted that TOTVS, a large Brazilian tech company, faced a surplus of enterprise data that was unsuitable for AI agents because existing transactional systems and data lakes were built for traditional applications or analyst dashboards. This mismatch creates latency and token-consumption problems when agents issue unpredictable, high-frequency queries. The need to re-architect the data layer is therefore the first concrete change discussed.

To address the mismatch, TOTVS is moving toward a data mesh architecture that distributes data ownership and enables low-latency database access for AI agents. Semantic ontologies are introduced to provide richer, machine-readable context, and a dynamic MCP (model-context-prompt) selection process tailors the size of context windows to the token budget. These components together aim to cut token overhead while keeping response times within acceptable bounds for real-time reasoning loops.

A central theme of the talk is the balance between deterministic software, which offers high precision, and non-deterministic LLMs, which introduce probabilistic reasoning. The decision of where to draw the line is framed as a formula involving three variables: precision, security, and cost. Engineers must evaluate each data access path against these variables to decide which parts of a workflow remain deterministic and which can be delegated to generative AI.

Implementing this architecture incurs costs in terms of added infrastructure complexity, such as deploying mesh nodes, integrating semantic layers, and managing dynamic MCP tooling. It also requires ongoing monitoring of token usage to stay within budgetary constraints. The approach may not be suitable for use cases demanding absolute 99.99% precision or where latency cannot be compromised, limiting its applicability to certain high-risk transactional scenarios.