RunLocal

Hardware-aware · Open source · Updated weekly

Local AI: run open source models on your own computer.

The hub for local LLMs, install guides and a hardware picker.

You can run AI models like the ones behind ChatGPT or Claude directly on your laptop or desktop, without sending your data anywhere. RunLocal shows you which open source model to choose for your hardware, which free software to install (Ollama, LM Studio, llama.cpp), and how to get started in about ten minutes. No prior knowledge required.

New to this? Start with the glossary for plain-language definitions, or jump straight to the hardware picker to see what your computer can run.

Models worth your disk space

A short list, opinionated. Full catalog in the directory.

See all models →

Qwen 3.8

Alibaba (Qwen team) · China

2026
License
Apache 2.0 on the 27B; custom Qwen licence on the 2.4T MoE
Context
262k tokens
Sizes
27B dense · 2.4T-A95B MoE
CodingMultimodal workflowsMultilingualLocal workstations

August 2026, and the most-liked open-weight release of the summer. The 27B is dense, Apache 2.0, natively multimodal, and had GGUF builds from unsloth and bartowski within days — which makes it the current default for a 24 GB card. Prism ML's Bonsai 2 build squeezes the same 27B into about 6 GB, provided you run it on their llama.cpp fork. The 2.4T MoE sibling is frontier-scale and carries a different licence.

DeepSeek V4.1 Flash

DeepSeek AI · China

2026
License
MIT
Context
1M tokens
Sizes
552B MoE backbone (8B active in prefill, 16B in decode) plus a 196B Engram memory
Agentic codingLong documentsMultimodal workflowsHigh-memory workstations

October 2026: the DeepSeek 4.1 generation, followed shortly by the 1017 checkpoint listed here. A new architecture rather than a V4 refresh: natively multimodal, a causal encoder-decoder design that cuts the KV cache to about a quarter of V4 Flash, and a 196B Engram memory read by sparse lookup. unsloth's GGUF files run from about 250 GB on disk (UD-IQ1_M) to 501 GB (UD-Q4_K_XL), Engram included, and unsloth says the 1-bit build runs on a machine with 128 GB of RAM or more. Not in the picker: running it needs unsloth's llama.cpp pull request, not yet in a release. Until it is, DeepSeek V4 Flash 0731 is the DeepSeek that stock llama.cpp runs.

DeepSeek V4 Flash

DeepSeek AI · China

2026
License
MIT
Context
1M tokens (65k native, extended 16x with YaRN)
Sizes
304B-A13B MoE (284B before the DSpark decoder)
Agentic codingLong documentsHigh-memory workstations

Superseded on capability by DeepSeek V4.1 Flash in October 2026, but still the DeepSeek that runs on stock llama.cpp, and the largest entry here that a single machine can still load. 256 routed experts with 6 active per token means roughly 13B parameters fire on any given token, so it answers far faster than 304B suggests. The 0731 checkpoint supersedes the June preview and carries a DSpark speculative-decoding module, which is where the extra weights over the 284B base come from. Weights ship natively in FP8, so a Q8 build costs only about 7 GB more than Q4.

GLM-5.3-Flash

Z.ai (Zhipu) · China

2026
License
MIT
Context
1M tokens (three linear-attention layers for every sparse one)
Sizes
321B MoE (18B active)
Agentic codingMultimodal workflowsHigh-memory workstations

Late August 2026, and the first natively multimodal GLM. A new base model rather than a GLM-5.2 derivative, MIT licensed, and at 5 million downloads the most pulled GLM on the Hub. unsloth's GGUF builds run from about 98 GB (UD-IQ1_M) through 109 GB (UD-Q2_K_XL) to 200 GB (UD-Q4_K_XL), which puts it in the same 128 GB class as DeepSeek V4 Flash. Not in the picker yet: the hybrid attention needs a llama.cpp pull request that has not reached a release, and we do not print commands for a build you would have to assemble by hand.

Snapshot · Nov 16, 2026

Trending on Hugging Face

Auto-curated from the Hugging Face Hub by a weighted mix of downloads, community likes and recency. Refreshes weekly via GitHub Action. License tier is a quick visual hint, not legal advice.

Ternary-Bonsai-2-27B-gguf

prism-ml · 27B

Permissive

Apache 2.0

Downloads
1.4M
Likes
3.0k
Updated
11 days ago
View on HF →score 0.85

MiniCPM5-2B

openbmb · 2B

Permissive

Apache 2.0

Downloads
1.0M
Likes
1.9k
Updated
7 days ago
View on HF →score 0.83

GLM-5.3

zai-org

Open weight

Custom license

Downloads
1.7M
Likes
2.5k
Updated
1 months ago
View on HF →score 0.82

DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic

xtools-at · 9B

Permissive

Apache 2.0

reasoningmathfunction-calling
Downloads
29.6k
Likes
7.8k
Updated
3 days ago
View on HF →score 0.81

MiniMax-M2.5

MiniMaxAI

Open weight

Custom license

Downloads
1.0M
Likes
1.5k
Updated
21 days ago
View on HF →score 0.80

humanizer

jialinyyzz · 12B

Permissive

Apache 2.0

Downloads
408.6k
Likes
2.3k
Updated
1 months ago
View on HF →score 0.79

LFM3-1.7B

LiquidAI · 1.7B

Open weight

Custom license

Downloads
107.0k
Likes
1.8k
Updated
6 days ago
View on HF →score 0.77

MiniCPM5-1B

openbmb · 1B

Permissive

Apache 2.0

Downloads
219.8k
Likes
1.2k
Updated
7 days ago
View on HF →score 0.77

The tools that actually run them

Runtimes, GUIs and inference servers, with their real trade-offs.

See all tools →

Ollama

Runtime

MIT

The fastest way to get a local LLM running with one command.

macOSLinuxWindows

Strengths

  • One-line install, one-line model pulls
  • Built-in OpenAI-compatible API on localhost:11434
  • Active model library with 4,500+ tagged variants

Trade-offs

  • Less raw throughput than vLLM under heavy concurrent load
  • Configuration is opinionated; advanced tuning means dropping into llama.cpp anyway

llama.cpp

Runtime

MIT

Maximum control and the broadest hardware coverage in the open ecosystem.

macOSLinuxWindowsAndroidiOS

Strengths

  • Runs almost anywhere: CUDA, ROCm, Metal, Vulkan, CPU-only
  • Tight GGUF quantization control
  • Reference implementation behind most desktop LLM tools

Trade-offs

  • Command-line first; the UX assumes you read READMEs
  • Quantization options multiply quickly, easy to pick the wrong one

DwarfStar (ds4)

Runtime

MIT

Running DeepSeek V4 Flash at usable speed on one 128 GB machine, and deliberately little else.

macOSLinux

Strengths

  • Asymmetric 2-bit quantization of the routed MoE experts only, which is what puts a 300B-class model inside 128 GB
  • Metal, CUDA and ROCm backends, with benchmarks published per machine rather than as a single headline number
  • On-disk KV cache, so a long session survives a restart

Trade-offs

  • Wants 96-128 GB of RAM to hold the Q2 build in memory, 256 GB or more for Q4; below that you are on the slower SSD-streaming path
  • Runs one model family on purpose: DeepSeek V4 Flash and PRO, plus GLM 5.2
  • The author calls it beta quality and the code changes fast
Visit project →No guide yet

LM Studio

GUI

Proprietary

Browsing, comparing and chatting with local models in a desktop GUI.

macOSLinuxWindows

Strengths

  • Polished chat UI with side-by-side model comparison
  • Built-in Hugging Face model browser
  • Local OpenAI-compatible API server with one click

Trade-offs

  • Closed source; the engine is llama.cpp but the shell is not
  • Less scriptable than CLI-first tools

Install guides

Latest writing

Why bother running AI locally?

The big cloud services are easier to start with. But there are real reasons to do it yourself. Three of them.

Your data stays with you

What you type and what the model answers never leave your computer. Handy when you are working with personal notes, client documents, internal code, or anything you would not paste into a public website.

It works even when the cloud does not

The model file lives on your disk. If the company that made it shuts down, raises prices, or simply changes its terms, your setup keeps working. The model you download today still runs in 2030 if your computer does.

No surprise bills

Cloud AI charges per use. Local AI costs you the price of your computer, plus electricity. After the first month, the marginal cost of an extra question is essentially zero.