Polars

Polars

Polars is a high-performance DataFrame library written in Rust with Python, R, and Node.js bindings. It uses a columnar memory layout (Apache Arrow), a lazy execution engine, and aggressive parallelism to outperform Pandas by 5–100x on many operations.

Key Features

• Lazy API: Build a query plan and let Polars optimize + execute it — ideal for large datasets
• Multi-threaded: Automatically parallelizes operations across all CPU cores
• Streaming mode: Process datasets larger than RAM in fixed-memory chunks
• Expressive expressions: A composable, chainable expression syntax with no index ambiguity
• Arrow-native: Zero-copy interop with PyArrow, DuckDB, pandas, and Hugging Face datasets
• Native I/O: Read/write Parquet, CSV, JSON, IPC, Avro, databases, and cloud storage

Quick Start

pip install polars

import polars as pl

df = pl.read_parquet("events.parquet")

result = (
    df.lazy()
    .filter(pl.col("event_type") == "purchase")
    .group_by("user_id")
    .agg(pl.col("amount").sum().alias("total_spent"))
    .sort("total_spent", descending=True)
    .limit(100)
    .collect()
)

Add to ai-supply

npx ai-supply add polars-dataframe-library

Curated mirror of the open-source Polars (MIT). Get it from the source.