AI tools

Official inference framework for 1-bit LLMs

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

Go ahead and axolotl questions

Run frontier AI locally.

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

LLM inference in C/C++

Distribute and run LLMs with a single file.

MLX: An array framework for Apple silicon

Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

SGLang is a high-performance serving framework for large language models and multimodal models.

Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

A high-throughput and memory-efficient inference and serving engine for LLMs