localai/localai

https://github.com/go-skynet/LocalAI/fork https://github.com/go-skynet/LocalAI/stargazers https://github.com/go-skynet/LocalAI/pulls https://github.com/go-skynet/LocalAI/releases

https://hub.docker.com/r/localai/localai https://quay.io/repository/go-skynet/local-ai?tab=tags&tag=latest

:bulb: Get help - ❓FAQ https://github.com/go-skynet/LocalAI/discussions :speech_balloon: *** :book: Documentation website

💻 Quickstart 🖼️ Models https://github.com/mudler/LocalAI/issues?q=is%3Aissue+is%3Aopen+label%3Aroadmap 🥽 Demo 🌍 Explorer https://github.com/mudler/LocalAI-examples

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that's compatible with OpenAI (Elevenlabs, Anthropic... ) API specifications for local AI inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families. Does not require GPU. It is created and maintained by https://github.com/mudler.

📚🆕 Local Stack Family

🆕 LocalAI is now part of a comprehensive suite of AI tools designed to work together:

Screenshots

💻 Quickstart

Run the installer script:

bash
curl https://localai.io/install.sh | sh

Or run with docker:

CPU only image:

bash
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-cpu

Nvidia GPU:

bash
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

CPU and GPU image (bigger size):

bash
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

AIO images (it will pre-download a set of models ready for use, see [***]

bash
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu

To load models:

bash
# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
local-ai run llama-3.2-1b-instruct:q4_k_m
# Start LocalAI with the phi-2 model directly from huggingface
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# Install and run a model from the Ollama OCI registry
local-ai run ollama://gemma:2b
# Run a model from a configuration file
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
# Install and run a model from a standard OCI registry (e.g., Docker Hub)
local-ai run oci://localai/phi-2:latest

For more information, see 💻 Getting started