Phi-3-mini-4k-instruct

Phi-3-mini-4k-instruct

Phi-3-mini-4k-instruct is a 3.8-billion-parameter small language model (SLM) developed by Microsoft Research and released under the MIT license. It is instruction-tuned for chat and task completion, yet lightweight enough to run locally on a laptop CPU or edge device.

Key features

• 3.8B parameters — deployable on CPU, mobile (iOS/Android via ONNX/llama.cpp)
• 4k token context window (8k variant also available)
• Instruction-following quality comparable to much larger models
• Available in GGUF, ONNX, and full precision flavors
• Ideal for edge deployment, privacy-sensitive on-device AI

Quick start

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "microsoft/Phi-3-mini-4k-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.float16, device_map="auto"
)

messages = [{"role": "user", "content": "Write a haiku about open-source AI."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Install via ai-supply

npx ai-supply add phi-3-mini-4k-instruct

Curated mirror of the open-source Phi-3-mini-4k-instruct (MIT). Get it from the source.