Mistral-7B-v0.1

Mistral-7B-v0.1

Mistral-7B-v0.1 is a 7-billion-parameter transformer language model released by Mistral AI under the Apache 2.0 license. Despite its compact size it outperforms Llama 2 13B across standard reasoning, math, and code benchmarks, making it the go-to open-weight base model for fine-tuning, RAG, and on-premise inference.

Key features

• Grouped-Query Attention (GQA) — faster inference, lower KV-cache memory
• Sliding Window Attention (SWA) — efficient handling of long contexts
• 32k token sliding context with 8k default
• 4-bit GPTQ and AWQ quantized variants available on HuggingFace
• Beats Llama 2 13B on MMLU, HumanEval, HellaSwag, and ARC

Quick start (transformers)

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "mistralai/Mistral-7B-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = "Explain transformers in one paragraph:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Install via ai-supply

npx ai-supply add mistral-7b-v01

Curated mirror of the open-source Mistral-7B-v0.1 (Apache-2.0). Get it from the source.