Grounded SAM — Open-Vocabulary Detection + Segmentation
Combines Grounding DINO and Segment Anything for text-prompt-driven object detection and precise segmentation in one pipeline.
Grounded Segment Anything
Grounded SAM marries Grounding DINO (open-vocabulary detection) with Segment Anything Model (SAM) to create a pipeline that can detect and precisely segment any object described in free-form text — no classes, no training.
Key Features
- Text-prompt detection via Grounding DINO: "a cat", "all vehicles", "the red cup"
- Pixel-perfect segmentation masks via SAM for each detected object
- Extensions: Stable Diffusion inpainting, RAM++ auto-tagging, Recognize Anything
- Grounded SAM 2 variant using SAM 2 for video object tracking+segmentation
- REST API and Gradio demo included
- Batch processing support for large image datasets
Quick Start
import groundingdino.datasets.transforms as T
from groundingdino.util.inference import load_model, predict
from segment_anything import sam_model_registry, SamPredictor
# 1) Detect with text prompt
model = load_model("groundingdino/config/GroundingDINO_SwinT.py", "weights/groundingdino_swint.pth")
boxes, logits, phrases = predict(model, image, caption="a cat", box_threshold=0.3, text_threshold=0.25)
# 2) Segment detected boxes
sampredictor = SamPredictor(sam_model_registry["vit_h"](checkpoint="weights/sam_vit_h.pth"))
sampredictor.set_image(image_np)
masks, _, _ = sampredictor.predict_torch(point_coords=None, point_labels=None, boxes=boxes)
Install via ai-supply
npx ai-supply add grounded-segment-anything-pipeline
Curated mirror of the open-source Grounded-Segment-Anything (Apache-2.0). Get it from the source.
Compromise signals — malicious or tampered code (leaked secrets, backdoors, a dropped executable) — reduce the score, and known dependency CVEs carry a bounded penalty (they warrant review but never QUARANTINE — update the dependency to clear). Other dangerous-by-capability traits are risk surface, expected for some capabilities. Every finding is mapped to its OWASP control below.
Findings mapped to the OWASP Top 10 for LLM Applications (2025) and the OWASP Machine Learning Security Top 10. Expand any flagged control for the exact findings — compromise reduces the score; expected/risk-surface do not, except a known CVE, which carries a small bounded penalty (high/critical → Review).
The same gate an agent runs before installing (POST /api/v1/trust/grounded-segment-anything-pipeline/check). Click a policy:
Consume Grounded SAM — Open-Vocabulary Detection + Segmentation programmatically. Authenticate with an API key or session — see Authorize an agent.
# Agents: CHECK BEFORE YOU INSTALL (no auth) — score, grade, level, capability manifest
curl https://ai-supply.store/api/v1/trust/grounded-segment-anything-pipeline
# Gate against your org policy (returns { pass, violations })
curl -X POST https://ai-supply.store/api/v1/trust/grounded-segment-anything-pipeline/check \
-H "Content-Type: application/json" \
-d '{"minGrade":"B","denyPermissions":["shell"],"denyUnknownEgress":true}'
# CLI
npx ai-supply add grounded-segment-anything-pipeline
# REST (install → download)
curl -X POST https://ai-supply.store/api/v1/listings/grounded-segment-anything-pipeline/install \
-H "Authorization: Bearer $AIM_KEY"
# MCP tool
install_listing({ "slug": "grounded-segment-anything-pipeline" })OpenAPI spec →Curated mirror — latest upstream source. See the repository for tagged releases.