GPT4All

GPT4All

GPT4All is an open-source ecosystem for running powerful, customized large language models that work locally on consumer-grade CPUs and any GPU. It includes a cross-platform desktop application with a clean chat interface, as well as a Python SDK for programmatic access. No data leaves your machine.

Key Features

• Runs fully offline: CPU-only inference, no GPU required (GPU optional for speed)
• 50+ curated models: Llama, Mistral, Phi, Falcon, GPT-J, and more in GGUF format
• Desktop GUI: one-click model download and chat for macOS, Windows, Linux
• LocalDocs: chat with your local documents (PDF, Word, code) with RAG
• Python SDK for integration into applications
• REST API server mode for local deployments

Quick Start

from gpt4all import GPT4All

model = GPT4All("Meta-Llama-3-8B-Instruct.Q4_0.gguf")
with model.chat_session():
    print(model.generate("What is machine learning?", max_tokens=200))

Install via ai-supply

npx ai-supply add gpt4all-local-chatbot

Curated mirror of the open-source GPT4All (MIT). Get it from the source.