Just posted to my Forbes column about why the future of enterprise AI might actually be… smaller.
We all know large language models are powerful — but they’re also expensive, both financially and energetically. While consumers pay $20 a month for tools from OpenAI, the backend reality is that LLM queries can cost 10–100x more than a traditional search and consume significantly more electricity. AT&T thinks there’s a smarter way forward: small language models, or SLMs.
“We have small language models that are, of course, way cheaper to run — in many cases about 10% of the cost of large language models,” AT&T’s chief data and AI officer Andy Markus told me. “They’re super fast, and the accuracy rates that we’ve achieved… are about as accurate as the large language models.”
Instead of building giant, general-purpose systems, AT&T fine-tunes 4–7 billion parameter models on tightly scoped internal data — contracts, network logs, policies, transcripts — and deploys them for specific tasks like root-cause network analysis, fraud detection, and contract clause extraction. The payoff? Faster performance, lower cost, and a jump from 2X to 4X ROI on AI initiatives in just a year.
There’s also a macro angle here. With AI already consuming an estimated 1–2% of global electricity — and projected to rise — more efficient models could dramatically shrink that footprint. For Markus, who once coded with punch cards, this moment feels early: agentic AI is emerging, humans are still in the loop, but autonomy is increasing step by step.
Small, in this case, isn’t about thinking small. It’s about narrowing the problem, sharpening the data, and getting enterprise-grade intelligence at a fraction of the cost.