extract-llms-docs
Verified Safeby nirholas
Overview
Extracts, parses, and organizes documentation from websites (llms.txt, install.md) for AI agents, LLMs, and automation workflows, offering structured, agent-ready formats and generation tools.
Installation
cd mcp-server && pnpm install && pnpm startEnvironment Variables
- ADMIN_KEY
- ANTHROPIC_API_KEY
Security Notes
The application inherently performs network requests to arbitrary, user-provided URLs for web scraping and API interactions (e.g., GitHub, sitemaps, llms.txt files). While a `fetchWithTimeout` function is implemented to prevent hanging requests and `User-Agent` headers are set for identification, this type of functionality always carries an inherent risk of Server-Side Request Forgery (SSRF) if not meticulously controlled, though no obvious direct vulnerabilities were identified in the provided snippets. Rate limiting is robustly implemented via a sliding window algorithm and applied across critical API endpoints, which is a strong positive for preventing abuse. Environment variables (`ANTHROPIC_API_KEY`, `ADMIN_KEY`) are correctly used for sensitive information instead of hardcoding. No direct `eval` or other highly dangerous code patterns were found.
Similar Servers
gpt-researcher
The GPT Researcher MCP Server enables AI assistants to conduct comprehensive web research and generate detailed, factual, and unbiased reports. It supports multi-agent workflows, local document analysis, and integration with external tools via the Machine Conversation Protocol (MCP) for various research tasks.
DevDocs
DevDocs is a web crawling and content extraction platform designed to accelerate software development by converting documentation into LLM-ready formats for intelligent data querying and fine-tuning.
mcp-server
Provides a Model Context Protocol (MCP) server for AI agents to search and retrieve curated documentation for the Strands Agents framework, facilitating AI coding assistance.
blz
Provides fast, local documentation search and retrieval for AI agents, using llms.txt files for line-accurate citations.