LexGLUE — Legal Language Understanding Benchmark

LexGLUE — Legal Language Understanding Benchmark

LexGLUE is the legal analogue of GLUE/SuperGLUE — a comprehensive benchmark spanning seven legal NLP datasets and tasks. It standardizes evaluation across EURLEX (EU legislation classification), ECHR (court judgement prediction), LEDGAR (contract provision classification), SCOTUS (US Supreme Court decision area), ECtHR (article violation prediction), ContractNLI (contract NLI), and CaseHOLD (legal holding identification).

Key Features

• 7 legal NLP tasks in a single evaluation harness
• Covers EU and US jurisdictions across legislation, contracts, and case law
• HuggingFace Datasets integration for easy loading
• Leaderboard tracking state-of-the-art Legal-BERT, RoBERTa-legal, and other models
• CC-BY-4.0 dataset license with public reproducibility

Quick Start

from datasets import load_dataset

# Load EURLEX classification task
dataset = load_dataset("coastalcph/lex_glue", "eurlex")
print(dataset["train"][0]["text"][:200])
print(dataset["train"][0]["labels"])  # Multi-label list

# Load ECHR court judgement prediction
scotus = load_dataset("coastalcph/lex_glue", "scotus")
print(scotus["test"][0])

npx ai-supply add lex-glue-legal-benchmark

Curated mirror of the open-source LexGLUE (CC-BY-4.0). Get it from the source.