Inference engine

Tools tagged Inference engine. 6 tools.

Inference engine

C/C++ inference for GGUF models on any hardware.

Efficient inference and serving for open models.

Run large language models locally with one command.

Fast serving and structured generation engine.

Hugging Face's production LLM server.

High-throughput self-hosted LLM serving engine.