Cash in on the AI Boom by Renting Out Your Spare

What happened

If you own an at-home server, a gaming computer, or just a laptop that doesn’t get much love, listen up. You can now put that spare computing power to use and earn some passive income in the process. AI companies are hungry for more compute to run AI inference—the process of using a pre-trained model to respond to queries—and they’re willing to pay you for it.

Federico sees the potential hosts as people much like himself who have already invested in home servers, and he aims to make the process of selling spare compute as seamless for them as possible. “Sign up for the program, install an application,” Federico says. “All we want to do is run jobs on your machine when you tell us we’re allowed to.

The only thing we do is monitor the resource usage. ” With a large enough network of devices, the platform would have compute available whenever it’s needed. Privacy and security are primary concerns for such hosts. To reassure the users that their local data is secure, and that no malware will be downloaded to their devices, the team open-sourced their scheduling software.

“The node software is open source, so anyone can look at it, see what it does. All we want to do is run jobs on your machine when you tell us we’re allowed to,” Federico says. Far Labs’ Shazhaev explains that the company’s software is designed around a principle known as “least privilege”: granting both the host and the user the least access possible to accomplish the task.

Inference runs as an isolated workload with authenticated, encrypted communication and explicit limits on the GPU, CPU, memory, storage, and network resources it may use. Customers do not receive arbitrary access to the host machine, and providers can inspect resource use, pause the node, revoke access, and remove the software at any time. The protection also works in the other direction.

“This is quite a difficult issue from the science angle,” Shazhaev says. ” Evolving Edge’s Federico says this is an issue for the largest, state-of-the art AI models. But those are not always needed and are often not even preferred. “There are numerous companies, once they reach a certain scale, suddenly paying for tokens on a state-of-the-art frontier model [that] no longer makes sense for their needs,” he says.

“Instead, they are fine-tuning open-source models for specific tasks that they have in their business. These models don’t require anywhere near the resources that some of the state-of-the-art models do. ” Smaller, open-source models can often fit on a single user device. But if that fails, there are tools to split a single inference task over multiple GPUs or CPUs.

Why it matters

“Imagine Uber or Airbnb, but for AI inference computing tasks,” says Ilman Shazhaev, founder and CEO of Far Labs, based in Abu Dhabi. The AI boom has spurred on construction of massive data centers, often damaging local communities by raising electricity prices, straining local water resources, causing environmental damage and noise, and being just plain ugly.

Huge data centers are likely not going anywhere—training new frontier models and running AI models from leading companies will likely still be the purview of these behemoths. But now, several companies are providing AI inference on smaller, mostly open-source models. They are running inference on pre-existing computing power spread throughout homes and small businesses, and compensating owners. “Everyone thinks the only way to do it is data centers.

And data centers are extractive for the communities in which they’re built, and they don’t return services or taxes or much of anything to the people there. ” says John Federico, founder and CEO of Evolving Edge, in Austin, Texas. “The compute power is out there.

” The idea isn’t entirely new: From 1999 to 2020, a volunteer-based project called SETI@Home used spare computers to search for signs of extraterrestrial life in radio telescope data, for instance. But now, commercial companies are eager to use the same strategy. Shazhaev’s Far Labs is launching its platform Far AI in the coming weeks, while Federico’s Evolving Edge is currently in open beta.

Other companies, like Bless Network, Salad, and Gradient have started to provide similar platforms over the last year. Federico has been a computer hobbyist since youth, and he has amassed a whole server in his basement to run his projects.

“It just hit me one day, there’s all this talk about not having enough compute, and I just thought, well, 92 percent of the country has broadband, and you have people like me who have mini data centers in a closet,” he says.

What to watch

Workloads are segmented and only the minimum required information is exposed to an individual node. Sensitive enterprise workloads can be restricted to controlled hardware rather than routed through consumer devices. Massive data centers still have advantages from the user perspective: top of the line GPUs and CPUs, high speed networking, thick cables, and sophisticated cooling. User devices are usually less powerful, more varied, and less reliably connected to one another.