Your first local AI in 10 minutes

This is the guide where you actually install something. Two paths: the fast one, and the one I recommend long-term. Both are free.

The fast path: Ollama

Go to the Ollama website, download the installer for your computer, and install it like any normal program. On a Mac it's a normal app, on Windows a normal installer.

Once it's installed, open a terminal and type:

ollama run llama3.2

That's it. The first time, it downloads the model, which takes a few minutes depending on your internet. When it finishes, you'll see a prompt where you can type to the model directly. Try:

"Write a short email thanking a client for their payment."

That model, llama3.2, is a good general-purpose small model. When you're done, type /bye to exit.

You now have a working local AI. To give it a proper chat window instead of the terminal, install Open WebUI and connect it to Ollama, a well-documented one-time step.

The recommended path: LocalAI

LocalAI is what I recommend for a business, because it's a drop-in replacement for the big cloud APIs. Point existing tools at it and they keep working, except now the model is on your machine. It also runs on the processor without a GPU.

Install it with Docker (a tool that runs programs in tidy little containers), or use the plain installer on the LocalAI site:

docker run -p 8080:8080 --name local-ai -ti localai/localai:latest

That starts LocalAI. Then pick a model from its gallery, which is sorted by what your hardware can actually handle, so you can't easily pick something that won't run. Point Open WebUI at it, and you've got your private ChatGPT.

Your first real test

Ask it to summarise a couple of paragraphs of real business text you have lying around, an email or a proposal page. If it comes back with a decent summary, you're done. You own an AI, it runs offline, and nobody can see what you ask it.

When it doesn't work

The usual suspects, in order: not enough memory (pick a smaller model), a failed download (run it again), or a firewall blocking a port (run it on a machine you control). All solvable in minutes.

If you'd rather have this set up, secured, and pointed at your actual documents without reading a word of this, that's the service I run. I build private AI for businesses in Dubai on hardware they own, and the first conversation is free.