BitNet
Official inference framework for 1-bit LLMs
Written mainly in C++. Released under the MIT licence.
Free plan
github.com/microsoft/BitNet
What each tool is actually for, described plainly. No ratings, no scores, no invented benchmarks.
Local models and inference: Running and fine-tuning models on your own hardware, and the servers that make them fast enough to use.
Official inference framework for 1-bit LLMs
Written mainly in C++. Released under the MIT licence.
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Written mainly in Python. Released under the Apache-2.0 licence.
LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
Written mainly in Go. Released under the MIT licence.
Go ahead and axolotl questions
Written mainly in Python. Released under the Apache-2.0 licence.
Run frontier AI locally.
Written mainly in Python. Released under the Apache-2.0 licence.
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Written mainly in C++. Released under the MIT licence.
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
Written mainly in TypeScript.
Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
Written mainly in C++. Released under the AGPL-3.0 licence.
LLM inference in C/C++
Written mainly in C++. Released under the MIT licence.
Distribute and run LLMs with a single file.
Written mainly in C++.
MLX: An array framework for Apple silicon
Written mainly in C++. Released under the MIT licence.
Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Written mainly in Go. Released under the MIT licence.
SGLang is a high-performance serving framework for large language models and multimodal models.
Written mainly in Python. Released under the Apache-2.0 licence.
Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.
Written mainly in Python. Released under the AGPL-3.0 licence.
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Written mainly in Python. Released under the Apache-2.0 licence.
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.
Written mainly in Python. Released under the Apache-2.0 licence.
A high-throughput and memory-efficient inference and serving engine for LLMs
Written mainly in Python. Released under the Apache-2.0 licence.