How to run LLMs on your local machine?
Ollama: run LLMs on your own machine. No cloud, no API keys, no monthly bill. Here's how.
Endo
Cloud LLMs are fine... until they're not. Your prompts leave your machine, you pay per token, and the moment you build something real the API bill gets annoying. And if you just want to mess around with a model without an account, you're signing up for rate limits and privacy policies.
Ollama just runs the model locally. Download it, type one command, and you have an LLM answering on your own hardware. No account, no API key, no data leaving the room. It packages everything - the model weights, the runtime, the API - behind a single ollama run.
Why local wins:
- Privacy: prompts stay on your machine. Nothing to leak, nothing to subpoena.
- No latency surprises: no roundtrip to a datacenter. It answers as fast as your hardware allows.
- It's free: you already own the hardware. A 4B model runs on a potato.
- Pick your model: llama, qwen, gemma, whatever - swap by typing a different name.
Honest downsides:
- Hardware is the ceiling: a big model on a small machine will crawl. If you don't have the RAM/VRAM, the model you want is simply not available to you.
- You're the ops team: updates, disk space, broken installs - that's you now.
- Smaller selection than the cloud: you get open models, not the frontier ones.
Quick Start Guide
To get started with Ollama and run LLMs on your local machine, follow the steps below to install Ollama on your operating system.
- Visit the Ollama website and download the macOS installer.
- Open the downloaded
.dmgfile and drag the Ollama application to your Applications folder and launch it. - Thats it! Ollama should now be installed and running on your macOS machine. You can verify the installation by typing
ollama -vin your terminal.
After you verified the installation, you can run a model by typing:
ollama run <model_name>
Replace
<model_name>with the name of the model you want to use. (e.g.,ollama run qwen3:4b)
- Visit the Ollama website and download the Windows installer.
- Run the downloaded
.exefile and follow the installation wizard to install Ollama on your system. - Thats it! Ollama should now be installed and running on your Windows machine. You can verify the installation by typing
ollama -vin your terminal.
After you verified the installation, you can run a model by typing:
ollama run <model_name>
Replace
<model_name>with the name of the model you want to use. (e.g.,ollama run qwen3:4b)
- Run the following command in your terminal to download and install Ollama:
curl -fsSL https://ollama.com/install.sh | sh - Thats it! Ollama should now be installed and running on your Linux machine. You can verify the installation by typing
ollama -vin your terminal.
After you verified the installation, you can run a model by typing:
ollama run <model_name>
Replace
<model_name>with the name of the model you want to use. (e.g.,ollama run qwen3:4b)
- Run the following command in your terminal to download and install Ollama in Docker:
CPU only:docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
GPU support:docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama - Thats it! Ollama should now be installed and running on your machine inside Docker. You can verify the installation by typing
docker exec -it ollama ollama -vin your terminal.
After you verified the installation, you can run a model by typing:
docker exec -it ollama ollama run <model_name>
Replace
<model_name>with the name of the model you want to use. (e.g.,ollama run qwen3:4b)
If you want to try LLMs without handing over a card, Ollama is the easiest way in. One install, one command, done. The big models need big hardware - but for everything else, your own machine is enough.
Aditional resources: