mcp-evals
Verified Safeby wolfeidau
Overview
A Go library and CLI for evaluating Model Context Protocol (MCP) servers using Claude, employing an agentic evaluation loop and detailed grading across multiple dimensions.
Installation
mcp-evals run --config evals.yamlEnvironment Variables
- ANTHROPIC_API_KEY
Security Notes
The tool is designed to execute external commands specified in its configuration files (e.g., `mcp_server.command` and `mcp_server.args`). While this is necessary functionality for evaluating MCP servers, it means that if the configuration file is untrusted, the `mcp-evals` tool could execute arbitrary malicious commands on the host system. The `install.sh` script handles binary downloads and checksum verification appropriately. No hardcoded secrets, obfuscation, or direct shell `eval` calls are found in the Go source code.
Similar Servers
mcp-interviewer
A Python CLI tool to evaluate Model Context Protocol (MCP) servers for agentic use-cases, by inspecting capabilities, running functional tests, and providing LLM-as-a-judge evaluations.
loki-mcp
A Go-based server that implements the Model Context Protocol (MCP) to integrate Grafana Loki log querying capabilities with AI agents like Claude Desktop or n8n.
mcpbr
A benchmark runner for evaluating Model Context Protocol (MCP) servers by comparing LLM agent performance with and without MCP tools on software engineering tasks.
gemini-mcp-rs
A high-performance Rust MCP server that enables AI-driven tasks by wrapping the Gemini CLI, facilitating integration with MCP-compatible clients like Claude Code.