llamafile

llamafile

llamafile, developed by Mozilla, lets you distribute and run large language models as a single executable file using Cosmopolitan Libc. The same binary runs natively on macOS, Windows, Linux, FreeBSD, and NetBSD — no Python, no Docker, no dependencies. It bundles llama.cpp with a small local web UI and an OpenAI-compatible server.

Key Features

• Single binary: model weights + runtime in one .llamafile executable file
• Universal binary: one build, six operating systems, x86-64 and ARM64
• Built-in web UI: chat interface available at http://localhost:8080
• OpenAI-compatible API for easy integration
• GPU acceleration: CUDA, Metal, and ROCm support when available
• Zero install: just chmod +x and run

Quick Start

# Download a llamafile (e.g., Mistral)
curl -LO https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile/resolve/main/mistral-7b-instruct-v0.2.Q4_0.llamafile
chmod +x mistral-7b-instruct-v0.2.Q4_0.llamafile

# Run (opens browser + API server)
./mistral-7b-instruct-v0.2.Q4_0.llamafile

Install via ai-supply

npx ai-supply add llamafile-single-file-llm

Curated mirror of the open-source llamafile (Apache-2.0). Get it from the source.