⊜Fine-tuneLanguage & NLPFree
TRL (Transformer Reinforcement Learning)
Fine-tune LLMs with RLHF, PPO, DPO, SFT, and GRPO — the standard library for aligning language models.
インストール数380k
評価★ 4.8
レビュー127
TRL — Transformer Reinforcement Learning
TRL is a full-stack library by Hugging Face for training transformer language models with reinforcement learning from human feedback (RLHF) and related alignment techniques. It provides efficient trainers for every stage of the modern LLM alignment pipeline.
Key Features
- SFTTrainer: supervised fine-tuning with packing, LoRA, and chat templates
- DPO/IPO/KTO: direct preference optimization variants — no reward model needed
- PPO: proximal policy optimization with reward model for classic RLHF
- GRPO: group relative policy optimization (as used in DeepSeek-R1)
- RewardTrainer: train reward models from preference data
- Integrates with PEFT, Accelerate, bitsandbytes for efficient training
- 🤗 Hub model card generation and W&B/TensorBoard logging
Quick Start
from trl import SFTTrainer, SFTConfig
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B")
trainer = SFTTrainer(
model=model,
args=SFTConfig(output_dir="/tmp/sft"),
train_dataset=dataset,
)
trainer.train()
Install via ai-supply
npx ai-supply add trl-rlhf-training
Curated mirror of the open-source TRL (Apache-2.0). Get it from the source.