unclecode/crawl4ai

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper.

Crawl4AI is the #1 trending GitHub repository, actively maintained by a vibrant community. It delivers blazing-fast, AI-ready web crawling tailored for LLMs, AI agents, and data pipelines. Open source, flexible, and built for real-time performance, Crawl4AI empowers developers with unmatched speed, precision, and deployment ease.

🧪 Try It

bash
docker run -d \
  -p 11235:11235 \
  --name crawl4ai \
  --shm-size=3g \
  unclecode/crawl4ai:latest

Now open http://localhost:***/playground a UI for testing crawls, tweaking configs, and generating JSON for your agents.

🔥 Sample API Call

bash
curl -X POST http://localhost:11235/crawl \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com"]}'

Or stream results:

bash
curl -N -X POST http://localhost:11235/crawl/stream \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com"], "crawler_config": {"type": "CrawlerRunConfig", "params": {"stream": true}}}'

📦 From Docker Hub

bash
docker pull unclecode/crawl4ai:latest
# Or specific version
docker pull unclecode/crawl4ai:0.6.0rc1-r1

Multi-arch support (amd64, arm64), .llm.env support for OpenAI, Claude, Groq, and others.

📚 Docs

Full docs at docs.crawl4ai.com – learn how to tweak configs, use browser profiles, write custom filters, or deploy at scale.

❤️ Support the Project

• ⭐ https://github.com/unclecode/crawl4ai
• 🧠 Follow @unclecode on X
• 💬 Join our ***

🖖 Live Long and import Crawl4ai