ScrapeGraphAI
Prompt-driven web scraping: describe the data you want and LLM graph pipelines extract clean structured JSON from any page.
ScrapeGraphAI
ScrapeGraphAI turns web scraping into a prompt. Instead of writing brittle CSS/XPath selectors, you point it at a URL (or local HTML/XML/JSON/Markdown) and describe the data you want in natural language; a graph of LLM-powered nodes fetches, parses, and returns clean structured output. It works with OpenAI, Anthropic, Groq, Azure, Gemini, and local Ollama models, so extraction logic survives the page redesigns that break traditional scrapers.
Key features
- Prompt-to-JSON extraction via composable scraping "graphs" (SmartScraper, SearchGraph, OmniScraper)
- Model-agnostic: cloud LLMs or fully local models through Ollama
- Handles single pages, multi-page search, and multimedia sources
- Built-in browser rendering for JavaScript-heavy sites
- Python API plus integrations for pipelines and agents
Ideal for research agents, dataset collection, and monitoring tasks where target sites change often and hand-tuned selectors are too costly to maintain.
Curated mirror of the open-source ScrapeGraphAI (MIT). Get it from the source.
Compromise signals — malicious or tampered code (leaked secrets, backdoors, a dropped executable) — reduce the score, and known dependency CVEs carry a bounded penalty (they warrant review but never QUARANTINE — update the dependency to clear). Other dangerous-by-capability traits are risk surface, expected for some capabilities. Every finding is mapped to its OWASP control below.
Findings mapped to the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any flagged control for the exact findings — compromise reduces the score; expected/risk-surface do not, except a known CVE, which carries a small bounded penalty (high/critical → Review).
The same gate an agent runs before installing (POST /api/v1/trust/scrapegraph-ai-llm-scraper/check). Click a policy:
Consume ScrapeGraphAI programmatically. Authenticate with an API key or session — see Authorize an agent.
# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/scrapegraph-ai-llm-scraper
# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/scrapegraph-ai-llm-scraper/check \
-H "Content-Type: application/json" \
-d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'
# CLI
npx ai-supply add scrapegraph-ai-llm-scraper
# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/scrapegraph-ai-llm-scraper/install \
-H "Authorization: Bearer $AIM_KEY"
# MCP tool
install_listing({ "slug": "scrapegraph-ai-llm-scraper" })OpenAPI spec →Curated mirror — latest upstream source. See the repository for tagged releases.