How to Run a Chatbot on Your Own Computer

What happened

If you're on Windows, a dedicated Nvidia GPU helps a lot: Graphics chips are better for running AI processes than standard processors, which is partly why Nvidia and the AI boom are so closely interlinked. These dedicated graphics cards will also have their own RAM, which gives AI models extra thinking space.

Smaller models won't be quite as capable, but they will download faster and take up less room on your system. Photograph: David Nield Back in the chat interface, everything works similar to a standard AI chatbot app.

In the prompt box at the bottom you can choose between your AI models (if you have more than one LLM installed), and on the left of the prompt box you've got a + (plus) icon for submitting images and files (if this is supported by the current model). On the left of the interface is the navigation pane, where you can switch between your projects.

Why it matters

So while there's no “minimum spec” for running a local LLM, as much RAM as possible and a discrete graphics card will certainly help. You then need a piece of software to run the model (to act as the outer interface), plus the model itself. There are plenty of choices for both, so you can pick and choose as required.

Photograph: David Nield For those getting started, LM Studio Bionic is generally considered to be the best AI app choice on Windows and macOS, and it’s free to use. cpp, Ollama, and GPT4All, which are again available for multiple operating systems. These pieces of software are a little more technical and involved, though. With an AI app chosen, the other decision is to settle on an LLM.

The software programs we've mentioned will guide you toward some options, and there are also online repositories of models, the most well-known being Hugging Face (which has more than 3 million models available—quite the selection).

A whole host of setups are possible when it comes to local LLMs, but we'll take you step by step through the process of setting up LM Studio Bionic on Windows, to give you an idea of how it works. Once you've downloaded the installation file from the website and have LM Studio Bionic up and running, click Create Project to get started, and give your first project a name.

A blank conversation window opens: If you click Choose a model in the prompt box, then Get local models, you'll be taken to the main model picker for LM Studio Bionic. Each of the available models is listed together with its size, its popularity, and a few details about it—keep an eye out for the staff picks if you want some recommendations to get started with.

What to watch

Via the Settings link (lower left) on the navigation pane you've got the preferences screen for LM Studio Bionic. There are a bunch of ways to customize the software here, from how deleted chats are handled to the interface elements onscreen. Photograph: David Nield Click Library in Settings to manage your existing models or the Explore menu entry to find new ones.

If you need support for images and documents, look for multimodal models. ) The final part of the LM Studio interface that is important to know about is the right-hand sidebar, which can be shown or hidden via the button on the very top-right corner. These give you further options for managing files across projects and giving the program access to the file system on your computer, if needed.