A agentindex
Tools · Inference engine

llama.cpp

C/C++ inference for GGUF models on any hardware.

Visit site GitHub
Open source Self-hostable free
Inference engine
In these stacks
Write code with AIBuild my own agent
LicenseMIT
PriceFree
UsageNo model cost
Model supportlocal, multi
Languagescpp
GitHub stars119.2k
Last activity2026-07-03
Verified2026-06-27
Verified byseed

A lightweight C/C++ inference engine and the GGUF model format, enabling LLM inference on CPUs and low-end GPUs across platforms.

Strengths

  • Runs on almost any hardware, including CPU-only
  • Defines the widely-used GGUF format

Tradeoffs

  • Lower-level than managed servers
  • Performance tuning requires quantization knowledge

Compare with