Chonkie

Chonkie

Chunking is the unglamorous step that quietly decides RAG quality — split too coarsely and retrieval drags in noise, too finely and you shatter context. Chonkie is a no-nonsense library dedicated to doing exactly this one job well, with a tiny footprint and fast defaults instead of a heavyweight framework.

Key features

• Multiple chunker strategies: token, word, sentence, recursive, semantic (embedding-similarity), and code-aware splitting
• Built for speed and low memory so ingestion of large corpora stays cheap
• Pluggable tokenizers and embedding backends; overlap and size fully configurable
• Handy SemanticChunker groups semantically coherent sentences for cleaner retrieval units
• Small dependency surface — installs light and drops into any ingestion pipeline

It slots in ahead of your embedding + vector-store step, turning raw documents into well-formed passages that downstream retrievers and rerankers can actually use.

Curated mirror of the open-source Chonkie (MIT). Get it from the source.