The standard single-file format for running quantized open-weight LLMs locally with llama.cpp, Ollama and LM Studio.
GGUF packages a model's weights, tokenizer and metadata into one file, pre-quantized to a chosen precision (Q4_K_M is the common default — roughly half a byte per parameter). It's what you actually download when you run an open-weight model locally through llama.cpp, Ollama or LM Studio.
The quantization level in the filename is a size/quality dial: Q2–Q3 shrink furthest with visible quality loss, Q4–Q5 are the sweet spot for most hardware, Q8 is near-lossless at twice the size. A 9B model is ~5–6GB at Q4 — comfortable on a 16GB machine.
GGUF is CPU-and-GPU friendly and runs on ordinary hardware, which is what makes local LLMs practical at all — no inference server, no per-token cost, full privacy.