Back to Home
scalvert icon

server-tester

Verified Safe

by scalvert

Overview

A Playwright-based testing framework for Model Context Protocol (MCP) servers, enabling validation of server behavior, tool functionality, protocol conformance, and LLM-driven evaluations.

Installation

Run Command
No command provided

Environment Variables

  • OPENAI_API_KEY
  • ANTHROPIC_API_KEY
  • MCP_ACCESS_TOKEN
  • MCP_REFRESH_TOKEN
  • MCP_TOKEN_TYPE
  • MCP_TOKEN_EXPIRES_AT
  • MCP_OAUTH_SERVER_URL
  • MCP_AUTH_STATE_PATH
  • MCP_OAUTH_CLIENT_ID
  • MCP_OAUTH_CLIENT_SECRET
  • MCP_OAUTH_SCOPES
  • MCP_OAUTH_RESOURCE
  • TEST_USER
  • TEST_PASSWORD
  • OAUTH_DEBUG

Security Notes

The project is a testing framework, not a server itself. It's designed to interact with external MCP servers via stdio (executing local commands) or HTTP. Potential risks are managed responsibly: - **Command Execution (stdio transport)**: The framework executes commands (e.g., `node server.js`, `npx ...`) as configured in `playwright.config.ts`. This is a high-privilege operation, but the commands are explicitly defined by the developer in the configuration, not derived from untrusted user input. - **OAuth Token Storage**: Sensitive OAuth tokens and client information are stored locally (default in user's home directory under `.local/state/mcp-tests` on Linux/macOS, `%LOCALAPPDATA%\mcp-tests` on Windows) with restrictive file permissions (`0o600` for files, `0o700` for directories) to prevent unauthorized access. - **Automated OAuth Login**: The `performOAuthSetup` utility automates browser-based OAuth login, which requires test user credentials (e.g., `TEST_USER`, `TEST_PASSWORD`). This is a powerful feature intended for controlled test environments. - **Dynamic Imports**: Uses dynamic imports for LLM SDKs (`openai`, `anthropic`) and the `open` package, but these are for known, specified dependencies, not arbitrary code execution. - **JSON Parsing**: JSON parsing is used for configuration files, test data, and tool responses, all within expected and controlled contexts. Overall, the project is considered safe to run, provided its configurations (especially command execution for stdio transport) are set up responsibly by the developer.

Similar Servers

Stats

Interest Score14
Security Score9
Cost ClassMedium
Avg Tokens750
Stars3
Forks0
Last Update2025-12-18

Tags

MCPtestingPlaywrightQALLMevaluationconformancetooling