What is the GGUF file format?
A .gguf file is a GGML Universal Format File, the binary container used by llama.cpp to store a large language model. Georgi Gerganov and the ggml community introduced it in August 2023 as the successor to the older GGML, GGMF and GGJT formats. A single .gguf file holds the model weights, the tokenizer and every hyperparameter as typed key-value metadata, so no separate configuration files are needed to load and run the model.
The format is built to be self-contained, extensible and memory-mappable. Because metadata is stored as namespaced key-value pairs instead of a fixed list, new fields can be added without breaking older readers. Weights are usually quantized, from 2-bit up to 8-bit and including K-quants such as Q4_K_M, which lets sizeable models run on consumer CPUs and GPUs. Every file starts with the four-byte magic string GGUF, followed by a version number; the current version is 3.
Security & safety
RISK: LOWGGUF is a data container for model weights and metadata, not an executable, so opening one does not run code by itself. The main practical risks are large file sizes and downloading models from untrusted sources; a malformed file could in theory exploit a bug in the loader, so use up-to-date inference tools and reputable model repositories.
Format details
in a nutshell- Genome Graphical User Format - Rarely, .gguf has been used ad hoc by hobby projects for GUI layout data, unrelated to the machine-learning format described here.
Programs that open GGUF files
Windows6 apps
LM Studio
Freemium Browse, download and chat with .gguf models through a desktop GUI, with per-model quantization choices.
Ollama
Open-source Load a local .gguf file via a Modelfile and serve it for chat or API use with one command.
llama.cpp
Open-source Run inference directly on a .gguf model with llama-cli or llama-server, the reference implementation of the format.
GPT4All
Open-source Add a downloaded .gguf model to the local model folder and chat with it offline.
koboldcpp
Open-source Load a .gguf model for text generation and roleplay with a self-contained executable. 
macOS4 apps
LM Studio
Freemium Download and run .gguf models with Apple Silicon acceleration through a desktop GUI.
Ollama
Open-source Serve a local .gguf model via a Modelfile using Metal acceleration on Apple hardware. 
Linux4 apps
llama.cpp
Open-source Build from source and run .gguf models with CUDA, ROCm or CPU backends via llama-cli and llama-server.
text-generation-webui
Open-source Load .gguf models through the llama.cpp loader in a browser-based interface.
Hugging Face GGUF viewer
Free Inspect a .gguf file's metadata, tensor list and quantization in the browser without downloading it. Technical details
deep spec| Encoding | Binary |
| Byte order | Little-endian by default; big-endian supported since version 3 |
| Container | Single-file container holding header, metadata key-value block, tensor info block and raw tensor data |
| Compression | None as a container; weights are stored using GGUF quantization schemes (Q2 through Q8, K-quants such as Q4_K_M, IQ variants, plus F16/BF16/F32) |
| Typical size | Roughly 0.5 GB to over 100 GB depending on parameter count and quantization level |
| Structure | Header (magic, version, tensor count, metadata count) then namespaced key-value metadata, then per-tensor info (name, dimensions, type, offset), then aligned raw tensor data. Default alignment is 32 bytes and must be a multiple of 8. |
| Integrity | No built-in checksum; the metadata block is self-describing and validated at load time |
| Platforms | Windows, macOS, Linux |
| Notes | A GGUF file is self-contained: it bundles the model weights, tokenizer and all hyperparameters as typed metadata, so no separate config files are needed. Metadata keys use lower_snake_case with period-separated namespaces and keys can be up to 65,535 bytes. Over 40 quantization types are supported. Large models can be sharded across multiple .gguf files with llama-gguf-split. |
| Released | August 2023 |
| Latest version | Version 3 |
| Open standard | Yes · royalty-free |
| Specification | github.com |
GGUF conversions
Community Q&A
asked by usersNo questions yet - be the first to ask about GGUF files.
Frequently asked questions
How do I open a GGUF file?
llama.cpp. These tools read the .gguf file and let you chat with the model. Double-clicking the file will not do anything useful on its own.What program creates GGUF files?
llama.cpp conversion scripts, mainly convert_hf_to_gguf.py, which turns a Hugging Face model into a .gguf, optionally followed by llama-quantize to shrink it to a smaller quantization such as Q4_K_M.What does the Q4_K_M in a GGUF filename mean?
Can I convert a GGUF file back to the original model?
.safetensors or PyTorch weights with community scripts, but because most GGUF files are quantized, the conversion is lossy and does not restore the exact original weights.Why is my GGUF file so large?
.gguf holds every model weight in one file. File size scales with the number of parameters and the quantization level, so a large model at higher precision can easily exceed tens of gigabytes.
