Unstructured
Open-source document ingestion library — partition PDFs, HTML, DOCX, and 25+ formats into clean elements for RAG pipelines.
Unstructured
Unstructured is the leading open-source library for ingesting and preprocessing unstructured documents for use in LLM applications. It partitions documents into typed elements (Title, NarrativeText, Table, Image, etc.), extracts metadata, and cleans text — making raw files ready for chunking and embedding.
Key Features
- 25+ file formats — PDF, HTML, DOCX, PPTX, XLSX, EML, MSG, MD, RST, TXT, images, and more
- Partition strategies —
fast(rule-based),hi_res(layout detection with detectron2),ocr_only - Element types — Title, NarrativeText, Table, ListItem, Image, Header, Footer, FigureCaption
- Table extraction — HTML table extraction from PDFs with
hi_resstrategy - LangChain / LlamaIndex connectors —
UnstructuredLoaderis a first-class integration in both - Connectors — S3, GCS, Azure Blob, Confluence, Google Drive, SharePoint, Slack, and more
Quick Start
pip install unstructured[pdf]
from unstructured.partition.pdf import partition_pdf
elements = partition_pdf("research_paper.pdf", strategy="hi_res")
for el in elements[:5]:
print(el.category, ":", str(el)[:80])
Install via ai-supply
npx ai-supply add unstructured-document-ingestion
Curated mirror of the open-source Unstructured project (Apache-2.0). Install upstream from the repository.
Compromise signals — malicious or tampered code (leaked secrets, backdoors, a dropped executable) — reduce the score, and known dependency CVEs carry a bounded penalty (they warrant review but never QUARANTINE — update the dependency to clear). Other dangerous-by-capability traits are risk surface, expected for some capabilities. Every finding is mapped to its OWASP control below.
Findings mapped to the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any flagged control for the exact findings — compromise reduces the score; expected/risk-surface do not, except a known CVE, which carries a small bounded penalty (high/critical → Review).
The same gate an agent runs before installing (POST /api/v1/trust/unstructured-document-ingestion/check). Click a policy:
Consume Unstructured programmatically. Authenticate with an API key or session — see Authorize an agent.
# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/unstructured-document-ingestion
# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/unstructured-document-ingestion/check \
-H "Content-Type: application/json" \
-d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'
# CLI
npx ai-supply add unstructured-document-ingestion
# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/unstructured-document-ingestion/install \
-H "Authorization: Bearer $AIM_KEY"
# MCP tool
install_listing({ "slug": "unstructured-document-ingestion" })OpenAPI spec →Curated mirror — latest upstream source. See the repository for tagged releases.