crawl4ai-mcp-server
Verified Safeby amienbou121
Overview
Provides a self-hosted web scraping and crawling server, similar to Firecrawl's API, for integration with AI frameworks and agents.
Installation
docker run --rm -i -v $(pwd)/crawls:/app/crawls -e CRAWL4AI_MCP_LOG=INFO uysalsadi/crawl4ai-mcp-server:latestEnvironment Variables
- CRAWL4AI_MCP_LOG
- TARGET_URL
- OPENAI_API_KEY
Security Notes
The server includes safety measures like blocking private/localhost URLs via `safety.py` to prevent access to internal networks. Output directories for persistence are created with a unique `run_id`, mitigating direct path traversal outside the designated mount point, but disk space exhaustion is still a potential risk if external input controls output volume. The `docker-compose.yml` uses `network_mode: host` for its main service, which reduces network isolation by sharing the host's network stack with the container, presenting a security concern if the host runs sensitive services or the container is exposed. No direct `eval` or obvious command injection vulnerabilities were found in the provided Python code.
Similar Servers
gpt-researcher
The GPT Researcher MCP Server enables AI assistants (like Claude) to conduct comprehensive, in-depth web research and generate detailed, curated reports via the Machine Conversation Protocol (MCP), providing higher quality, optimized context, and better reasoning compared to standard search tools.
Scrapling
Enables AI chatbots and agents to perform adaptive web scraping, extract targeted data, and bypass anti-bot protections conversationally.
firecrawl-mcp-server
Provides web scraping, crawling, search, and structured data extraction capabilities to AI models via the Model Context Protocol.
mcp-server-browserbase
Enables LLMs to perform cloud browser automation tasks such as navigating, interacting with elements, extracting data, and capturing screenshots on web pages.