AT&T now processes about 45 billion AI tokens a day. The useful lesson is not the size of that number. It is the architecture behind it: several models, retrieval systems and fixed workflows operating under one control layer.
In July, AT&T added H2O AI Super Agent across customer experience, fraud prevention, field operations, enterprise research and automation. The name suggests one all-purpose digital worker. The implementation points in the other direction.
The orchestration layer breaks an objective into smaller jobs, sends each job to a suitable component and adjusts the sequence when context changes. A research agent can search internal material. A predictive model can score fraud risk. A fine-tuned small language model can classify routine text. A larger reasoning model only sees the cases that justify its cost.
That is how we build production AI systems. Start with the work, not the model.
The economics are already visible in an earlier AT&T call-centre system. AT&T divided 80 classification categories between a conventional classifier, a fine-tuned 1.8 billion parameter H2O Danube model and Llama 3.1 70B. The combined pipeline reached 91% accuracy at 35% of the previous OpenAI cost.
It also cut daily processing time from 15 hours to 4.5 hours. Transcript capacity rose from 45,000 to 250,000. Those gains did not come from finding a smarter universal model. They came from matching each category to the cheapest tool that could handle it reliably.
For a mid-market business, the same pattern works at a smaller scale. Map one workflow first. Separate predictable steps from ambiguous ones, then route them deliberately. A rules engine may validate an invoice total. A small local model can classify documents or redact personal data. Retrieval can supply approved company knowledge. A frontier model can handle the few cases that need deeper reasoning. People take the exceptions.
The control layer matters more than the model catalogue. Every agent needs its own identity, narrow permissions, spend limits and a complete action log. Do not let an agent borrow a staff member's broad access token. CrowdStrike has warned that a compromised agent can inherit a user's privileges across many systems, while a Cloud Security Alliance survey found 68% of respondents could not reliably distinguish agent actions from human activity.
Governance also changes adoption. Info-Tech Research Group surveyed 551 senior leaders and found measurable, department-wide AI impact at 60% of firms with a governed strategy, compared with 20% of firms without one. AT&T paired its Microsoft 365 Copilot rollout with more than 200 live training sessions, embedded AI ambassadors and daily usage monitoring. It reached 96.4% sustained adoption.
The next system we deploy should have a routing table before it has an agent name: which tasks run locally, which call a larger model, which use fixed automation, and exactly when a person takes over.
Leave a Reply