Macs don’t just do AI, they’re replacing the cloud for it

08/09 19:15 - Macs don’t just do AI, they’re replacing the cloud for it
I surprised myself this morning when I came across an interesting Apple-commissioned report — Rethinking critical AI infrastructure — I’d not seen before. It looks at the shifting expectations for AI infrastructure and recognizes that enterprise users want (and need) secure, on-device AI solutions for critical parts of their business. That’s why tens of thousands of companies are already investing in Macs, because they recognize that Macs do indeed do AI. The study, published earlier this year and put together by Omdia, reflects insights gathered across 1,500 conversations with enterprise tech leaders and practitioners, noting that for many in business the current cloud-based approach to AI fails to deliver on three key metrics: Costs: Current pricing models seem unsustainable. Particularly when it comes to agentic AI, costs climb fast and business users need to get those costs under control.  Security: Even the most secure cloud services include some degree of data risk. When it comes to using AI for regulated data in industries such as healthcare, business users need much more security than the cloud inherently provides. After all, data that is not transmitted will not leak in transmission. >Capacity>: Workload requirements change and capacity needs to scale. That can boost the cost of accessing additional cloud capacity, or impose limitations in the event it can’t be found. It’s also true that while frontier models can provide all the bells and whistles of AI for advanced tasks, the vast majority of the AI work does not require anything near as much power. As Omdia explains: “57% of enterprise models are under 10 billion parameters, well within the capabilities of modern devices like MacBook Air or the entry-level MacBook Pro.” What they think As you might expect, the researchers believe on-premises AI set-ups respond to all three needs; not only that, but once you’ve coughed up cash for the necessary computational infrastructure, you don’t have to pay much more. “On-device infrastructure has near-zero marginal cost after initial investment, enabling unlimited experimentation without budget constraints,” the report said.  Basically, once you’ve invested in on-premises capacity, you can divert mundane AI tasks to those machines for processing — limiting costs, boosting security and releasing capacity, turning to cloud-based models only when higher end AI solutions are required. While that’s good news for Apple, that’s bad news for many AI companies’ revenue models. (Perhaps they should have recognized that even the most advanced LLM’s will run on a standard iPhone eventually.) The other advantage is that if AI is not used as widely as expected across a company, the same hardware can be used for other company tasks. What’s actually happening Enterprises already using AI are learning these lessons, which is why we see more of them buying Macs for these tasks. They do so because Apple’s computers deliver the computational power and performance to run AI effectively, from chip design to power consumption to the OS itself. Apple has intentionally built its platforms to be the best in class for running AI on device, and the Unified Memory architecture Apple has created in Apple Silicon scales really well, meaning you can run ever larger LLMs on Macs.  It’s not just Macs, either. An iPad can run up to 14 billion parameter models quite happily; a Mac Studio reaches 480 billion; and a cluster of four Mac Studios will take you all the way to 1.6 trillion parameters using off-the-shelf cables. To put that into context, Omdia found that 57% of the AI models typically used by the enterprise come in at under 10 billion parameters, which implies that enterprises could run a huge chunk of their AI tasks on an iPad, an iPhone, and certainly on a Mac. The ability of Apple’s ecosystem to scale is precisely why most AI developers at frontier model companies already use Macs. “Organizations that build AI solutions in-house adopt Mac for AI workloads at nearly double the rate of organizations buying commercial solutions,” the report explained. The takeaway Apple is emerging as an important component of an overall ecosystem for applied AI in the enterprise — or anywhere else — challenging frontier models with a scalable, controllable, economical, and secure approach to deployed AI that delivers most of the bang expected for the enterprise buck.  While Apple paid for the report, that doesn’t necessarily invalidate its conclusions, which are not myopic around the Apple platform. Apple does not replace everything else, it just becomes one of the pillars to build success with AI. Companies can use other AI services and solutions, but they’ll want Macs along for at least some of the ride. And as the models themselves evolve and become slimmer and more refined, the platforms that run them best will deliver the advantage business users need. Now, we need Apple to develop tools for the management, deployment, and governance of these solutions. Please subscribe to my daily, human-curated Apple-related news headline feed at The Core, or follow me on BlueSky, LinkedIn, or Mastodon. ...