Crafter
Procedurally generated open-world survival benchmark that evaluates a broad range of RL and world-model agent abilities.
Crafter
Crafter is an open-world survival game designed as a benchmark for reinforcement-learning and world-model research, created by Danijar Hafner. Each episode is a procedurally generated 2D world where an agent must forage, craft tools, build shelter, and survive threats — evaluating a broad spectrum of agent abilities from a single environment.
Key features
- Procedurally generated worlds with diverse biomes, resources, creatures, and day/night cycles
- 22 semantically meaningful achievements (collect wood, make pickaxe, defeat zombie) used to score general capability
- Fast, lightweight rendering suitable for large-scale training on modest hardware
- Gym-compatible API with pixel observations and a discrete action space
- Standardized reward and success metrics for reproducible comparison across methods
Because progress requires exploration, generalization, long-horizon reasoning, and a deep crafting tech-tree, Crafter has become a popular testbed for evaluating sample-efficient and unsupervised agents.
Curated mirror of the open-source Crafter (MIT). Get it from the source.
Compromise signals — malicious or tampered code (leaked secrets, backdoors, a dropped executable) — reduce the score, and known dependency CVEs carry a bounded penalty (they warrant review but never QUARANTINE — update the dependency to clear). Other dangerous-by-capability traits are risk surface, expected for some capabilities. Every finding is mapped to its OWASP control below.
Findings mapped to the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any flagged control for the exact findings — compromise reduces the score; expected/risk-surface do not, except a known CVE, which carries a small bounded penalty (high/critical → Review).
The same gate an agent runs before installing (POST /api/v1/trust/crafter-open-world-benchmark/check). Click a policy:
Consume Crafter programmatically. Authenticate with an API key or session — see Authorize an agent.
# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/crafter-open-world-benchmark
# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/crafter-open-world-benchmark/check \
-H "Content-Type: application/json" \
-d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'
# CLI
npx ai-supply add crafter-open-world-benchmark
# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/crafter-open-world-benchmark/install \
-H "Authorization: Bearer $AIM_KEY"
# MCP tool
install_listing({ "slug": "crafter-open-world-benchmark" })OpenAPI spec →Curated mirror — latest upstream source. See the repository for tagged releases.