Introduction

What if you could ask an AI assistant to run a test suite, explain a failing scenario, or summarize a complex error log in plain English without changing your existing test automation framework?

Modern automation frameworks such as Pytest and Playwright are incredibly powerful, but they’re often designed with technical users in mind. Running tests, configuring execution parameters, and investigating failures can require considerable framework knowledge, especially as projects grow in size and complexity.

I wanted to explore whether AI could make those workflows more accessible without compromising the flexibility and reliability of the underlying framework.

The result was a proof-of-concept architecture that combines Pytest, Playwright, and the Model Context Protocol (MCP). By introducing a lightweight MCP layer, both technical and non-technical users can interact with a test automation framework using natural language, while the framework itself remains fully independent and capable of running through its normal CLI and CI/CD workflows.

Why I Built It

The idea originated from several recurring challenges I have encountered in test automation projects.

Challenge 1: Complex Test Execution

As automation frameworks grow, test execution commands can become increasingly complex.

A simple test run may require:

Environment selection
Brand or tenant configuration
Browser settings
Parallel execution options
Custom pytest flags
Report generation settings

While experienced engineers can navigate these commands, they still create friction during day-to-day work.

Challenge 2: Failure Analysis Takes Time

When an E2E test fails, the failure itself is often not the biggest problem.

The real effort comes from:

Reviewing logs
Reading stack traces
Understanding screenshots
Analyzing traces
Identifying likely root causes

Many failures contain valuable information, but extracting that information quickly remains a challenge.

Challenge 3: Limited Accessibility for Non-Technical Stakeholders

Automated testing delivers significant business value, but access to that value is often limited to engineering teams. Product owners, business analysts, support teams, and other stakeholders frequently need answers to questions such as:

Did this user journey still work?
Can we execute this scenario?
What failed?
What changed?

Traditional test automation tooling is not designed with these users in mind.

The Goal

The primary objective was not to replace Pytest or Playwright.

Instead, I wanted to:

Keep the automation framework fully independent
Preserve existing CI/CD workflows
Add a lightweight AI interface layer
Support both technical and non-technical users
Explore practical MCP integrations

A key design principle was that the framework should still function perfectly without AI. The MCP layer should act as an enhancement, not a dependency.

The Architecture

For a more technical view please visit the MCP-Pytest-Blueprint Github repo.

The blueprint consists of two independent components:

1. Pytest + Playwright Framework

The automation framework remains responsible for:

Test execution
Browser automation
Reporting
Logging
Parallel execution
Environment management

It can still be executed entirely through the command line or CI/CD pipelines.

How to run the tests

The framework dynamically selects users, brands, and environments through configuration-driven execution, allowing a single test suite to scale across multiple brands and deployment environments.

Sequential:

uv run pytest -s tests/test_x.py
========================================
Captured Environment Name: test
Captured Brands Input:     all
Resulting List:            ['aaa', 'bbb', '...']
========================================
xdist_group_1: test_x[aaa] → test_x[bbb] → test_x[ccc] → test_x[ddd] → test_x[eee] → test_x[fff]

Parallel:

uv run pytest tests/test_x.py --env=test --brands=all -n auto --dist=loadgroup --headed
========================================
Captured Environment Name: test
Captured Brands Input:     all
Resulting List:            ['aaa', 'bbb', '...']
========================================
xdist_group_1 (abc_user): test_x[aaa] → test_x[bbb] → test_x[ccc]
xdist_group_2 (def_user): test_x[ddd] → test_x[eee] → test_x[fff]
                          └──────── run simultaneously ─────────┘

2. MCP Server Layer

The MCP server acts as a bridge between AI clients and the test framework.

This layer allows AI assistants to:

Trigger test runs
Filter tests
Read reports
Analyze failures
Explain results
Communicate in natural language

How to run the tests with a remote LLM

remote LLM example

How to run the tests with a local LLM

Alt text describing the image

Advantages & Considerations

Introducing an extra layer comes with both benefits and trade-offs:

Advantages

Natural language interaction with automated tests
Faster failure investigation and report analysis
Reduced learning curve for non-technical stakeholders
Improved accessibility across cross-functional teams
Existing Pytest and CI/CD workflows remain unchanged
Support for both local and remote LLMs

Considerations

Introduces an additional component that must be maintained
AI-generated explanations still require human validation
Additional security considerations may be required in enterprise environments
Not every project benefits from the added complexity
MCP tooling and integrations will continue to evolve

What I Learned

MCP Is Simpler Than It Appears

Before building the project, MCP felt like a complex new technology. In reality, the protocol itself is straightforward once you understand its role as a standardized interface between AI models and tools. The real challenge lies in designing meaningful tools and workflows.

Good Documentation Matters More Than Ever

AI systems depend heavily on context. Well-documented tools, clear descriptions, and predictable outputs dramatically improve the quality of interactions. Poor documentation leads to poor AI experiences.

Existing Frameworks Still Matter

Despite the excitement surrounding AI, the strongest solutions still rely on robust underlying systems.

The Pytest and Playwright foundation remains the most important component of the architecture. AI enhances the experience but does not replace good framework design.

AI Works Best as an Assistant

The goal is not autonomous testing.

The goal is reducing friction:

Less command memorization
Faster debugging
Easier reporting
Better collaboration

Final Thoughts

This project started as an experiment. I wanted to better understand how AI technologies could be combined with quality engineering practices without disrupting the foundations that make those practices reliable.

What began as an exploration of MCP quickly became a broader lesson in accessibility, usability, and collaboration. The most valuable outcome wasn’t simply connecting an AI model to a test framework, but discovering new ways to make testing information easier to access, understand, and act upon for both technical and non-technical stakeholders.

As AI continues to evolve, I believe some of the biggest opportunities won’t come from replacing existing tools and processes, but from making them more approachable and more effective. Strong frameworks, good engineering practices, and quality-focused teams will remain essential. AI simply provides another layer that can help reduce friction and improve how people interact with technology.

This blueprint represents one step in that journey. It allowed me to deepen my understanding of MCP, explore practical AI integrations, and think differently about how quality engineering can evolve in the years ahead.

And like many good side projects, it answered a few questions while creating many new ones to explore next.

Additional Resources

Interested in learning more about this project?

How to use AI for Test Automation: Building the MCP-Pytest Framework