MCP Conformance Test Framework

A framework for testing MCP (Model Context Protocol) client and server implementations against the specification.

> [!WARNING]
> This repository is a work in progress and is unstable. Join the conversation in the #conformance-testing-wg in the MCP Contributors discord.

For SDK maintainers: See SDK Integration Guide for a streamlined guide on integrating conformance tests into your SDK repository.

Quick Start

Testing Clients

# Using the everything-client (recommended)
npx @modelcontextprotocol/conformance client --command "tsx examples/clients/typescript/everything-client.ts" --scenario initialize
Run an entire suite of tests

npx @modelcontextprotocol/conformance client --command "tsx examples/clients/typescript/everything-client.ts" --suite auth

Testing Servers

# Run all server scenarios (default)
npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp
Run a single scenario

npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp --scenario server-initialize

List Available Scenarios

npx @modelcontextprotocol/conformance list

Overview

The conformance test framework validates MCP implementations by:

For Clients:

Starting a test server for the specified scenario

Running the client implementation with the test server URL

Capturing MCP protocol interactions

Running conformance checks against the specification

Generating detailed test results

For Servers:

Connecting to the running server as an MCP client

Sending test requests and capturing responses

Running conformance checks against server behavior

Generating detailed test results

Usage

Client Testing

npx @modelcontextprotocol/conformance client --command "<client-command>" --scenario <scenario-name> [options]

Options:

--command - The command to run your MCP client (can include flags)

--scenario - The test scenario to run (e.g., "initialize")

--suite - Run a suite of tests in parallel (e.g., "auth")

--expected-failures <path> - Path to YAML baseline file of known failures (see Expected Failures)

--timeout - Timeout in milliseconds (default: 30000)

--verbose - Show verbose output

The framework appends <server-url> as an argument to your command and sets the MCP_CONFORMANCE_SCENARIO environment variable to the scenario name. For scenarios that require additional context (e.g., client credentials), the MCP_CONFORMANCE_CONTEXT environment variable contains a JSON object with scenario-specific data.

Server Testing

npx @modelcontextprotocol/conformance server --url <url> [--scenario <scenario>]

Options:

--url - URL of the server to test

--scenario <scenario> - Test scenario to run (e.g., "server-initialize"). Runs all available scenarios by default

--suite <suite> - Suite to run: "ac

... [truncated — view full README on GitHub]