DevDocs
Verified Safeby cyberagiinc
Overview
DevDocs is a web crawling and content extraction platform designed to accelerate software development by converting documentation into LLM-ready formats for intelligent data querying and fine-tuning.
Installation
git clone https://github.com/cyberagiinc/DevDocs.git && cd DevDocs && cp .env.template .env && ./docker-start.shEnvironment Variables
- NEXT_PUBLIC_BACKEND_URL
- BACKEND_URL
- MCP_HOST
- CRAWL4AI_URL
- CRAWL4AI_API_TOKEN
- DISCOVERY_POLLING_TIMEOUT_SECONDS
- MAX_CONCURRENT_TASKS
- DISABLE_AUTH
- NODE_ENV
- PYTHONPATH
Security Notes
The server includes robust path traversal prevention for file content access. However, a default demo API key for Crawl4AI is hardcoded, which should be replaced in production environments. The use of `execPromise` to run shell scripts from an API endpoint (`/api/debug`) is present, but it's specifically for a contained debug script (`debug_crawl4ai.sh`), which reduces immediate risk within a self-hosted development context. The MCP server is executed with write access to `/app/storage/markdown`, making its integrity critical.
Similar Servers
mcp-omnisearch
Provides a unified interface for various search, AI response, content processing, and enhancement tools via Model Context Protocol (MCP).
mcp-server
Provides a Model Context Protocol (MCP) server for AI agents to search and retrieve curated documentation for the Strands Agents framework, facilitating AI coding assistance.
webscraping-ai-mcp-server
Integrates with WebScraping.AI to provide LLM-powered web data extraction, including question answering, structured data extraction, and HTML/text retrieval, with advanced features like JavaScript rendering and proxy management.
crawl-mcp
A comprehensive Model Context Protocol (MCP) server that wraps the crawl4ai library for advanced web crawling, content extraction, and AI-powered summarization from various sources including web pages, PDFs, Office documents, and YouTube videos.