crawl4ai-mcp-server
Verified Safeby amienbou121
Overview
A lightweight server providing web scraping and crawling tools, designed for integration with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code.
Installation
docker run --rm -i --volume /path/to/local/output:/app/crawls uysalsadi/crawl4ai-mcp-server:latestEnvironment Variables
- CRAWL4AI_MCP_LOG
- TARGET_URL
- OPENAI_API_KEY
Security Notes
The server explicitly implements safety measures in `crawler_agent/safety.py` to block non-public HTTP(S) URLs, including localhost, private IPs, and certain internal domains. It also mentions respecting `robots.txt` and enforcing domain policies. No direct 'eval' or obvious malicious patterns were found. The use of external web requests always carries inherent risk, but the codebase shows a good effort in mitigating common network-related vulnerabilities.
Similar Servers
gpt-researcher
The GPT Researcher MCP Server enables AI assistants to conduct comprehensive web research and generate detailed, factual, and unbiased reports. It supports multi-agent workflows, local document analysis, and integration with external tools via the Machine Conversation Protocol (MCP) for various research tasks.
Scrapling
Provides adaptive web scraping capabilities to AI chatbots and agents, allowing them to fetch, parse, and extract targeted data from websites, including dynamic content and anti-bot protected sites.
firecrawl-mcp-server
A Model Context Protocol (MCP) server for integrating Firecrawl's web scraping, crawling, search, and structured data extraction capabilities with AI agents.
mcp-server-browserbase
Enables LLMs to perform cloud browser automation tasks such as navigating, interacting with elements, extracting data, and capturing screenshots on web pages.