OpenOrca

OpenOrca

OpenOrca is a 4.2-million-row instruction fine-tuning dataset released under the MIT license by the Open-Orca community. It replicates and extends the Microsoft Orca research paper by generating chain-of-thought (CoT) explanation traces using GPT-4 and GPT-3.5, then attaching them to the FLAN collection. Models fine-tuned on OpenOrca consistently outperform same-size alternatives on reasoning and instruction-following tasks.

Key features

• 4.2M rows of system prompt + instruction + GPT-4/3.5 CoT response triples
• Covers FLAN-1M, CoT, and Open Platypus subsets
• MIT license — fully permissive for commercial fine-tuning
• Powers open-weight models like OpenHermes, MistralOrca, LLaMA-Orca
• Parquet format — fast loading with HuggingFace datasets

Quick start

from datasets import load_dataset

# Load a 10% sample to start
ds = load_dataset("Open-Orca/OpenOrca", split="train[:10%]")
print(ds[0])
# {'system_prompt': ..., 'question': ..., 'response': ...}

# Filter GPT-4 only rows for highest quality
gpt4_only = ds.filter(lambda x: "gpt-4" in x["id"])
print(f"{len(gpt4_only)} GPT-4 rows")

Install via ai-supply

npx ai-supply add open-orca-dataset

Curated mirror of the open-source OpenOrca (MIT). Get it from the source.