Apple’s ‘Minions’ tech turns idle Macs into an AI data center

21/10 13:30 - Apple’s ‘Minions’ tech turns idle Macs into an AI data center
If your business has lots of Macs, what do you do with them at night? And how much money do you spend on AI inferencing? What if the computational power of those Macs could be used to handle some inferencing tasks? A new white paper from Apple’s machine learning team tells us it’s possible. Apple’s teams have developed an internal technology for distributed processing they call “Minions,” which effectively creates a virtual GPU cloud from connected Macs. While it’s not available outside Apple at this time, it seems a pretty good candidate to turn Macs into a distributed AI data center, cutting the cost of AI inferencing and making it easier for enterprises to safely analyze company data by ensuring the information stays in the company.  Meet Apple’s Minions This exciting approach is described in a newly published white paper, ‘Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices.” In it, the researchers explain how constrained GPU resources delay access to inference tasks and increase costs — even while powerful Macs running Apple silicon chips may sit idle.  They’ve developed an approach to use that power. It’s based on three components: a Minion Server, an Executor, and a Watchdog. The server splits tasks up and puts them into a queue; the executor then acts as kind of a nightclub concierge, directing tasks across all the available Macs. The watchdog monitors active tasks and checks for failures — a Mac may be shut down or go to sleep during a task, for example. If a node goes offline, the watchdog returns that sub-task to the queue, making sure the work gets done. Minions currently supports two kinds of GPU inference on Macs, embedding generation and LLM completions using open-source transformer models from Hugging Face and Ollama, respectively. Ollama is installed and executed on participating devices, enabling local inference without remote API calls. The Apple team plans to extend Minions so it also supports distributed model training in the future, which should mean enterprise AI developers will be able to call on the power of all your available Macs to build new models as required. “The platform is particularly valuable in environments where centralized GPU resources are constrained, enabling teams to unlock substantial compute capacity at zero cost using existing hardware,” the paper notes. Inferencing power you already have in-house Of course, what’s most important about Minions is that it unleashes all that unused computational power without requiring people to spend way more money on external inference capacity, even when you factor in the electrical cost of running enough Macs to generate inferencing tokens. That’s potentially great news for any company working on AI projects that happens to have some Macs around with available capacity, and less great news for the big AI services firms who stand to lose another chunk of business from cloud to on-prem data handling.  AI inferencing capacity is in high demand, and Apple’s new internal tech should be an attractive way to use the computational power your company already has for machine learning/artificial intelligence or even (super?) intelligence tasks. Or, I guess, the “supra-hyper-bow-down-puny-mortal intelligence” some in Silicon Valley may be praying for. Ultimately, of course, any business using AI will be in a much more powerful position concerning its use if they own the means of production. Minions shows it’s possible, at least to some extent. No more ten-hour queues If you combine Minions with RDMA-clustered Macs running AI, then you end up in a very interesting place. The report tells us Apple has used the tech to make GPU compute more readily available to teams.  “Prior to its adoption, some inference jobs experienced queue times exceeding ten hours due to limited centralized GPU availability,” they said. “With Minions, tasks can often begin execution within minutes by leveraging idle macOS devices.” The white paper touches on a couple more topics worth mentioning: Temporary teams: “In practice, teams frequently coordinate executor participation informally. For example, team members may send Slack messages requesting others to temporarily run the Minion Executor and target a specific project or task,” they said. Confidentiality: To mitigate concerns over confidentiality, Minions supports team- or organization-scoped deployments “backed by dedicated PostgreSQL databases and isolated executor pools.” What that means is that when handling confidential data, organizations can create small pools of Macs to ensure strong risk governance. Will Apple stop there? It seems unlikely. The researchers wrote: “More broadly, Minions points toward a novel opportunity for the machine learning community: if idle macOS devices across organizations — or potentially acrossthe global community — can be securely federated, they could form a massive, shared GPU resource pool that benefits the broader ecosystem.” While some may blink and miss it, this news is potentially as exciting to the wider AI community as the capacity to connect Macs up for AI tasks using a Thunderbolt cable and still resonant introduction of the MLX framework became. Why build a data center when your AI project can shine with the power of 100 million Macs? The only flaw in this outpouring of enthusiasm is that Apple hasn’t yet announced any plans to make Minions available outside of the company itself. But we live in hope. Now please subscribe to my daily, human-curated Apple-related news headline feed at The Core, or follow me on BlueSky, LinkedIn, or Mastodon. ...