Project 35: Chrome Browser Testing Automation
Project 35: Chrome Browser Testing Automation
Build an agent that uses Chrome browser integration to perform visual regression testing, accessibility audits, and end-to-end user flow testing with AI-powered assertions that understand page content semantically.
Quick Reference
| Attribute | Value |
|---|---|
| Difficulty | Advanced (Level 3) |
| Time Estimate | 1-2 weeks |
| Primary Language | TypeScript |
| Alternative Languages | Python, JavaScript |
| Prerequisites | Projects 1-5, 20 (Web Scraper with Browser MCP) |
| Key Topics | Browser automation, visual regression, accessibility auditing, AI-powered assertions |
| Kiro CLI Concepts | Chrome MCP, computer use, screenshot analysis, hooks |
| Main Book Reference | “Web Application Testing with Puppeteer” by Jay Huang |
1. Learning Objectives
By completing this project, you will:
- Master Chrome browser automation through MCP: Control browser sessions, navigate pages, and interact with elements programmatically
- Implement visual regression testing: Capture screenshots, compare against baselines, and detect visual changes
- Build accessibility auditing systems: Analyze pages for WCAG compliance using DOM analysis and AI interpretation
- Create AI-powered assertions: Use natural language to describe expected behaviors and let AI verify them
- Generate comprehensive test reports: Produce human-readable reports with screenshots, findings, and recommendations
2. Deep Theoretical Foundation
2.1 The Evolution of Browser Testing
Browser testing has evolved through several paradigms, each building on the previous:
BROWSER TESTING EVOLUTION
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ GENERATION 1: Record and Playback │
│ ───────────────────────────────── │
│ │
│ • Selenium IDE, record mouse clicks │
│ • Brittle - breaks with any UI change │
│ • No understanding of what's being tested │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ GENERATION 2: Programmatic Automation │
│ ───────────────────────────────────── │
│ │
│ • Selenium WebDriver, Puppeteer, Playwright │
│ • Code-based selectors (CSS, XPath) │
│ • Still requires explicit assertions │
│ • Maintenance burden with UI changes │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ GENERATION 3: Visual Testing │
│ ──────────────────────────── │
│ │
│ • Percy, Chromatic, Applitools │
│ • Screenshot comparison │
│ • Catches visual regressions │
│ • Can't verify behavior │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ GENERATION 4: AI-Powered Testing (THIS PROJECT) │
│ ──────────────────────────────────────────────── │
│ │
│ • Natural language test descriptions │
│ • Semantic understanding of pages │
│ • Self-healing selectors │
│ • Intelligent assertions │
│ │
│ Example: │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ "Verify the checkout button is prominently displayed │ │
│ │ and enabled when items are in the cart" │ │
│ │ │ │
│ │ AI interprets: │ │
│ │ • Find checkout button (semantic search, not CSS selector) │ │
│ │ • Check visibility and prominence (size, color, position) │ │
│ │ • Verify enabled state (not disabled, clickable) │ │
│ │ • Check cart state (items present in cart indicator) │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
2.2 Chrome DevTools Protocol (CDP)
The Chrome MCP uses CDP to control the browser. Understanding CDP is essential:
CHROME DEVTOOLS PROTOCOL ARCHITECTURE
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ KIRO CLI CHROME MCP CHROME │
│ ──────── ────────── ────── │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ │ Commands │ │ CDP │ │ │
│ │ Agent │ ────────────▶ │ MCP │ ────────────▶ │ Browser │ │
│ │ │ │ Server │ │ │ │
│ │ │ ◀──────────── │ │ ◀──────────── │ │ │
│ │ │ Results │ │ Events │ │ │
│ └──────────┘ └──────────┘ └──────────┘ │
│ │
│ CDP DOMAINS (commonly used): │
│ ───────────────────────────── │
│ │
│ Page Domain │
│ ├── navigate(url) → Load a page │
│ ├── captureScreenshot() → Take screenshot │
│ └── getFrameTree() → Get frame structure │
│ │
│ DOM Domain │
│ ├── getDocument() → Get DOM tree │
│ ├── querySelector() → Find element │
│ └── getBoxModel() → Element dimensions │
│ │
│ Input Domain │
│ ├── dispatchMouseEvent() → Mouse actions │
│ ├── dispatchKeyEvent() → Keyboard input │
│ └── setFiles() → File upload │
│ │
│ Accessibility Domain │
│ ├── getFullAXTree() → Accessibility tree │
│ └── queryAXTree() → Query by role/name │
│ │
│ Runtime Domain │
│ ├── evaluate() → Execute JavaScript │
│ └── callFunctionOn() → Call function on object │
│ │
└─────────────────────────────────────────────────────────────────────────┘
2.3 Visual Regression Testing Patterns
Visual regression testing catches unintended visual changes:
VISUAL REGRESSION TESTING WORKFLOW
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ BASELINE CAPTURE │
│ ──────────────── │
│ │
│ 1. Navigate to page │
│ 2. Wait for stable state (no animations, loaded images) │
│ 3. Capture screenshot │
│ 4. Store as baseline with metadata │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Baseline: /products page │ │
│ │ Captured: 2024-12-22T10:30:00Z │ │
│ │ Viewport: 1920x1080 │ │
│ │ File: baselines/products-desktop.png │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ COMPARISON │
│ ────────── │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Baseline │ │ Current │ │ Diff │ │
│ │ │ │ │ │ │ │
│ │ ┌───────────┐ │ │ ┌───────────┐ │ │ ┌───────────┐ │ │
│ │ │ Header │ │ │ │ Header │ │ │ │ ░░░░░░░░░ │ │ │
│ │ ├───────────┤ │ vs │ ├───────────┤ │ = │ ├───────────┤ │ │
│ │ │ Products │ │ │ │ Products │ │ │ │ ████████ │ │ │
│ │ │ [A] [B] │ │ │ │ [A] [B] [C]│ │ │ │ [C]│ │ │
│ │ └───────────┘ │ │ └───────────┘ │ │ └───────────┘ │ │
│ │ │ │ │ │ New element! │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │
│ Diff Algorithm: │
│ • Pixel-by-pixel comparison │
│ • Perceptual hashing for fuzzy matching │
│ • Ignore regions for dynamic content │
│ • Tolerance threshold for anti-aliasing │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ DECISION MATRIX │
│ ─────────────── │
│ │
│ Diff Percentage Action Example │
│ ───────────────── ────────────────── ───────────────────────── │
│ 0% PASS No changes │
│ 0.1% - 1% REVIEW Minor shift, anti-aliasing │
│ 1% - 5% WARNING Layout change, new element │
│ > 5% FAIL Major redesign │
│ │
└─────────────────────────────────────────────────────────────────────────┘
2.4 Accessibility Auditing with WCAG
Web Content Accessibility Guidelines (WCAG) define standards for accessible web content:
WCAG COMPLIANCE LEVELS
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ LEVEL A (Minimum) │
│ ───────────────── │
│ │
│ Must-have accessibility features: │
│ • All images have alt text │
│ • Form inputs have labels │
│ • Links have descriptive text │
│ • Page has proper heading structure │
│ • Keyboard navigation works │
│ │
│ AI Check: │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ [A11y] Checking Level A compliance... │ │
│ │ ├── Images: 12 found, 2 missing alt text ✗ │ │
│ │ ├── Forms: 5 inputs, all labeled ✓ │ │
│ │ ├── Links: 23 found, 1 empty text ✗ │ │
│ │ ├── Headings: Proper h1→h2→h3 structure ✓ │ │
│ │ └── Keyboard: All interactive elements focusable ✓ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ LEVEL AA (Standard) │
│ ─────────────────── │
│ │
│ Enhanced accessibility: │
│ • Color contrast ratio ≥ 4.5:1 (normal text) │
│ • Color contrast ratio ≥ 3:1 (large text) │
│ • Text resizable to 200% without loss │
│ • Focus indicators visible │
│ • Error messages descriptive │
│ │
│ AI Check: │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ [A11y] Checking Level AA compliance... │ │
│ │ ├── Contrast: 3 elements below ratio ✗ │ │
│ │ │ └── .button-secondary: 2.8:1 (needs 4.5:1) │ │
│ │ ├── Text resize: Layout intact at 200% ✓ │ │
│ │ ├── Focus: All elements have visible focus ✓ │ │
│ │ └── Errors: 2 forms have generic error text ✗ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ LEVEL AAA (Enhanced) │
│ ──────────────────── │
│ │
│ Highest accessibility: │
│ • Color contrast ratio ≥ 7:1 (normal text) │
│ • Sign language for video content │
│ • Extended audio descriptions │
│ • Reading level appropriate │
│ │
└─────────────────────────────────────────────────────────────────────────┘
2.5 AI-Powered Assertions
Traditional assertions are brittle. AI-powered assertions understand intent:
TRADITIONAL VS AI-POWERED ASSERTIONS
┌─────────────────────────────────────────────────────────────────────────┐
│ │
│ TRADITIONAL ASSERTION │
│ ───────────────────── │
│ │
│ expect(page.locator('.btn-checkout')).toBeVisible() │
│ expect(page.locator('.btn-checkout')).toBeEnabled() │
│ expect(page.locator('.cart-count')).toHaveText('3') │
│ │
│ Problems: │
│ • Class name changes break tests │
│ • Doesn't verify visual prominence │
│ • Can't handle dynamic content │
│ • No semantic understanding │
│ │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ AI-POWERED ASSERTION │
│ ──────────────────── │
│ │
│ "The checkout button should be prominently displayed, │
│ enabled, and the cart should show items are present" │
│ │
│ AI Analysis: │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ 1. Find checkout button (semantic search) │ │
│ │ ├── Look for button with text containing "checkout" │ │
│ │ ├── Or button with aria-label related to checkout │ │
│ │ └── Found: <button class="primary-action">Checkout</button> │ │
│ │ │ │
│ │ 2. Verify "prominently displayed" │ │
│ │ ├── Size: 200x50px (larger than average button) │ │
│ │ ├── Position: Above fold, right side (common CTA position) │ │
│ │ ├── Color: High contrast #FF6B00 on white │ │
│ │ └── Result: PROMINENT ✓ │ │
│ │ │ │
│ │ 3. Verify "enabled" │ │
│ │ ├── Check: not [disabled] attribute │ │
│ │ ├── Check: aria-disabled !== "true" │ │
│ │ ├── Check: opacity > 0.5 │ │
│ │ └── Result: ENABLED ✓ │ │
│ │ │ │
│ │ 4. Verify "cart shows items" │ │
│ │ ├── Find cart indicator (icon with number/badge) │ │
│ │ ├── Check badge shows number > 0 │ │
│ │ └── Result: HAS ITEMS ✓ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
│ Benefits: │
│ • Self-healing (finds elements by intent, not selector) │
│ • Verifies visual aspects (prominence, visibility) │
│ • Understands semantic meaning │
│ • Natural language documentation │
│ │
└─────────────────────────────────────────────────────────────────────────┘
3. Complete Project Specification
3.1 What You Will Build
A Kiro CLI agent that:
- Controls Chrome browser sessions through MCP
- Executes user flows with natural language descriptions
- Performs visual regression testing with baseline comparison
- Runs accessibility audits against WCAG guidelines
- Uses AI to interpret pages and verify assertions
- Generates comprehensive HTML reports with screenshots
3.2 Functional Requirements
| Requirement | Description |
|---|---|
| Browser Control | Navigate, click, type, scroll through Chrome MCP |
| User Flow Testing | Execute multi-step test scenarios |
| Visual Regression | Compare screenshots against baselines |
| Accessibility Audit | Check WCAG A/AA compliance |
| AI Assertions | Verify expectations using natural language |
| Report Generation | Create HTML reports with findings |
| Baseline Management | Create, update, and manage visual baselines |
3.3 Non-Functional Requirements
- Reliability: Handle flaky tests with smart waits and retries
- Performance: Complete full test suite in reasonable time
- Maintainability: Natural language tests are self-documenting
- Portability: Tests work across different environments
4. Solution Architecture
4.1 System Architecture
┌──────────────────────────────────────────────────────────────────────────────┐
│ CHROME BROWSER TESTING AGENT │
├──────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ CLI INTERFACE │ │
│ │ │ │
│ │ kiro-cli chat │ │
│ │ > "Test the checkout flow on staging.example.com" │ │
│ │ │ │
│ └────────────────────────────────────┬────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ TEST ORCHESTRATOR │ │
│ │ │ │
│ │ ┌────────────────┐ ┌────────────────┐ ┌────────────────┐ │ │
│ │ │ Flow Parser │ │ Test Executor │ │ Result │ │ │
│ │ │ │ │ │ │ Aggregator │ │ │
│ │ │ • Parse steps │ │ • Run tests │ │ │ │ │
│ │ │ • Extract │ │ • Capture │ │ • Collect │ │ │
│ │ │ assertions │ │ screenshots │ │ results │ │ │
│ │ │ • Build plan │ │ • Verify │ │ • Generate │ │ │
│ │ │ │ │ assertions │ │ report │ │ │
│ │ └────────────────┘ └────────────────┘ └────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────┬──────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ TESTING COMPONENTS │ │
│ │ │ │
│ │ ┌────────────────┐ ┌────────────────┐ ┌────────────────┐ │ │
│ │ │ Visual Tester │ │ A11y Auditor │ │ AI Asserter │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ • Screenshot │ │ • DOM analysis │ │ • NL parsing │ │ │
│ │ │ • Baseline │ │ • WCAG checks │ │ • Screenshot │ │ │
│ │ │ comparison │ │ • AXE-core │ │ analysis │ │ │
│ │ │ • Diff report │ │ integration │ │ • Semantic │ │ │
│ │ │ │ │ │ │ verification │ │ │
│ │ └────────────────┘ └────────────────┘ └────────────────┘ │ │
│ │ │ │
│ └──────────────────────────────┬──────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ CHROME MCP │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │
│ │ │ Navigation │ │ Interaction │ │ Analysis │ │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ • Navigate │ │ • Click │ │ • Screenshot │ │ │
│ │ │ • Wait │ │ • Type │ │ • DOM tree │ │ │
│ │ │ • Reload │ │ • Scroll │ │ • A11y tree │ │ │
│ │ │ │ │ • Select │ │ • Evaluate │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────┘ │ │
│ │ │ │
│ └─────────────────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────────┘
4.2 Test Flow Data Structures
// Core types for browser testing
interface TestFlow {
name: string;
description: string;
baseUrl: string;
steps: TestStep[];
config: TestConfig;
}
interface TestStep {
id: string;
description: string; // Natural language description
action?: ActionSpec;
assertion?: AssertionSpec;
visual?: VisualSpec;
accessibility?: AccessibilitySpec;
}
interface ActionSpec {
type: 'navigate' | 'click' | 'type' | 'scroll' | 'wait' | 'select';
target?: string; // CSS selector, text, or natural language description
value?: string;
options?: {
timeout?: number;
waitFor?: 'visible' | 'hidden' | 'stable';
};
}
interface AssertionSpec {
type: 'ai' | 'element' | 'text' | 'url';
description: string; // Natural language for AI assertions
selector?: string;
expected?: string | RegExp;
}
interface VisualSpec {
type: 'full-page' | 'element' | 'viewport';
selector?: string;
baseline?: string;
threshold?: number; // 0-1, percentage of allowed difference
ignoreRegions?: { x: number; y: number; width: number; height: number }[];
}
interface AccessibilitySpec {
level: 'A' | 'AA' | 'AAA';
rules?: string[]; // Specific rules to check
exclude?: string[]; // Rules to skip
}
interface TestConfig {
viewport: { width: number; height: number };
timeout: number;
retries: number;
screenshotOnFailure: boolean;
recordVideo: boolean;
}
interface TestResult {
flow: string;
status: 'passed' | 'failed' | 'skipped';
duration: number;
steps: StepResult[];
screenshots: Screenshot[];
accessibility?: AccessibilityResult;
visual?: VisualResult;
}
interface StepResult {
step: TestStep;
status: 'passed' | 'failed' | 'skipped';
duration: number;
error?: string;
screenshot?: string;
aiAnalysis?: string;
}
interface VisualResult {
baseline: string;
current: string;
diff?: string;
diffPercentage: number;
status: 'match' | 'mismatch' | 'new-baseline';
}
interface AccessibilityResult {
level: 'A' | 'AA' | 'AAA';
score: number; // 0-100
issues: AccessibilityIssue[];
}
interface AccessibilityIssue {
rule: string;
impact: 'critical' | 'serious' | 'moderate' | 'minor';
element: string;
description: string;
fix: string;
}
5. Phased Implementation Guide
Phase 1: Browser Control Foundation (Days 1-3)
Goals:
- Establish Chrome MCP connection
- Implement basic navigation and interaction
- Capture screenshots
Tasks:
- Set up Chrome MCP integration
- Implement navigate, click, type actions
- Add screenshot capture
- Build wait/stability detection
- Create simple test runner
Milestone Checkpoint:
$ kiro-cli chat
> "Navigate to https://example.com and take a screenshot"
[Chrome MCP] Connecting to browser...
✓ Chrome session established
✓ Navigating to https://example.com
✓ Page loaded (1.2s)
✓ Screenshot captured: screenshot-20241222-103000.png
Hint 1: Use networkidle wait condition to ensure all resources are loaded before screenshots.
Phase 2: User Flow Testing (Days 4-6)
Goals:
- Parse natural language test descriptions
- Execute multi-step flows
- Report step-by-step results
Tasks:
- Create test flow parser
- Implement step execution engine
- Add error handling and retries
- Build result aggregation
- Create console progress output
Milestone Checkpoint:
> "Test the login flow:
1. Navigate to /login
2. Enter email 'test@example.com'
3. Enter password 'password123'
4. Click the login button
5. Verify redirect to dashboard"
[Test: Login Flow]
├── Step 1: Navigate to /login ✓
├── Step 2: Enter email 'test@example.com' ✓
├── Step 3: Enter password ✓
├── Step 4: Click login button ✓
└── Step 5: Verify redirect to dashboard ✓
Flow completed: 5/5 steps passed (2.3s)
Hint 2: For finding elements, try multiple strategies in order: exact text, aria-label, placeholder, then CSS selector.
Phase 3: Visual Regression (Days 7-9)
Goals:
- Implement baseline management
- Build image comparison
- Generate visual diff reports
Tasks:
- Create baseline storage system
- Implement pixel-by-pixel comparison
- Add perceptual diff algorithm
- Build ignore region support
- Generate HTML diff reports
Milestone Checkpoint:
> "Run visual regression on /products page"
[Visual Test: /products]
├── Baseline: baselines/products-desktop.png
├── Current: current/products-desktop.png
├── Comparing...
│
│ Diff: 3.2% (threshold: 1%)
│ ┌────────────────────────────────────────────────────────┐
│ │ Changes detected: │
│ │ • New element at (850, 320): "Sale" badge │
│ │ • Price changed: $29.99 → $24.99 │
│ │ • Layout shift: Product grid moved 5px right │
│ └────────────────────────────────────────────────────────┘
│
└── Status: FAILED (above threshold)
Generated: reports/visual-diff-products.html
Hint 3: Use a tolerance of 0.1% for anti-aliasing differences, but flag anything above 1% for human review.
Phase 4: Accessibility Auditing (Days 10-11)
Goals:
- Implement WCAG checks
- Integrate with accessibility tree
- Generate compliance reports
Tasks:
- Build accessibility tree analyzer
- Implement WCAG rule checks
- Add contrast ratio calculation
- Create issue categorization
- Generate accessibility report
Milestone Checkpoint:
> "Run accessibility audit on /checkout"
[Accessibility Audit: /checkout]
├── Level: AA
├── Score: 87/100
│
├── Issues Found: 6
│ ├── CRITICAL (1)
│ │ └── Form input missing label: #card-number
│ │
│ ├── SERIOUS (2)
│ │ ├── Contrast ratio 2.8:1 on .hint-text (needs 4.5:1)
│ │ └── Focus not visible on .link-terms
│ │
│ ├── MODERATE (2)
│ │ ├── Missing heading structure (h1 → h3, skipped h2)
│ │ └── Tab order not logical in address form
│ │
│ └── MINOR (1)
│ └── Image without alt: decorative-border.png
│
└── Report: reports/a11y-checkout.html
Phase 5: AI-Powered Assertions (Days 12-14)
Goals:
- Implement natural language assertion parsing
- Use AI to analyze screenshots
- Build semantic element finding
Tasks:
- Create assertion parser
- Implement screenshot analysis with AI
- Build semantic element finder
- Add confidence scoring
- Create detailed assertion reports
Milestone Checkpoint:
> "Verify the checkout button is prominent and enabled when cart has items"
[AI Assertion]
├── Analyzing page...
│
├── Finding checkout button:
│ └── Found: <button>Proceed to Checkout</button> at (1050, 450)
│
├── Checking prominence:
│ ├── Size: 200x48px (large for context) ✓
│ ├── Position: Above fold, primary action area ✓
│ ├── Color: High contrast #FF6B00 on white ✓
│ └── Result: PROMINENT ✓
│
├── Checking enabled state:
│ ├── Disabled attribute: false ✓
│ ├── Aria-disabled: not set ✓
│ ├── Visual opacity: 100% ✓
│ └── Result: ENABLED ✓
│
├── Checking cart state:
│ ├── Cart icon found at (1150, 25)
│ ├── Badge shows "3" items
│ └── Result: HAS ITEMS ✓
│
└── Assertion: PASSED ✓
6. Testing Strategy
6.1 Testing the Test Framework
describe('VisualComparison', () => {
it('should detect no difference for identical images', async () => {
const baseline = await loadImage('test-fixtures/page-a.png');
const current = await loadImage('test-fixtures/page-a.png');
const result = await compareImages(baseline, current);
expect(result.diffPercentage).toBe(0);
expect(result.status).toBe('match');
});
it('should detect differences above threshold', async () => {
const baseline = await loadImage('test-fixtures/page-a.png');
const current = await loadImage('test-fixtures/page-b.png');
const result = await compareImages(baseline, current, { threshold: 0.01 });
expect(result.diffPercentage).toBeGreaterThan(0.01);
expect(result.status).toBe('mismatch');
});
it('should ignore specified regions', async () => {
const baseline = await loadImage('test-fixtures/with-date.png');
const current = await loadImage('test-fixtures/with-different-date.png');
const result = await compareImages(baseline, current, {
ignoreRegions: [{ x: 10, y: 10, width: 100, height: 20 }]
});
expect(result.status).toBe('match');
});
});
6.2 Accessibility Audit Tests
describe('AccessibilityAuditor', () => {
it('should detect missing alt text', async () => {
const html = `
<img src="photo.jpg">
<img src="icon.png" alt="">
<img src="logo.png" alt="Company Logo">
`;
const issues = await auditAccessibility(html, { level: 'A' });
expect(issues).toContainEqual(expect.objectContaining({
rule: 'img-alt',
impact: 'critical'
}));
expect(issues.filter(i => i.rule === 'img-alt')).toHaveLength(2);
});
it('should check contrast ratios', async () => {
const html = `
<p style="color: #777; background: #fff;">Low contrast text</p>
`;
const issues = await auditAccessibility(html, { level: 'AA' });
expect(issues).toContainEqual(expect.objectContaining({
rule: 'color-contrast',
impact: 'serious'
}));
});
});
7. Common Pitfalls and Debugging
7.1 Common Issues
| Pitfall | Symptom | Solution |
|---|---|---|
| Flaky tests | Random failures | Add smart waits, use stable selectors |
| Screenshot timing | Captures mid-animation | Wait for networkidle + animation complete |
| Element not found | Selector breaks | Use semantic finding (text, role, label) |
| Baseline drift | Too many false positives | Update baselines intentionally, use ignore regions |
| Slow tests | Suite takes too long | Parallelize, reuse browser sessions |
7.2 Debugging Strategies
1. Enable verbose logging:
> "Run tests with debug output"
[DEBUG] Navigating to /products
[DEBUG] Waiting for networkidle...
[DEBUG] Network idle after 1.2s
[DEBUG] Finding element: "Add to Cart button"
[DEBUG] Strategy 1 (text): Found 0 matches
[DEBUG] Strategy 2 (aria-label): Found 3 matches
[DEBUG] Strategy 3 (role=button): Found 1 match (best fit)
[DEBUG] Clicking element at (450, 320)
2. Capture failure screenshots:
[FAILED] Step 3: Click checkout button
Failure details:
├── Error: Element not interactable
├── Element: <button disabled>Checkout</button>
├── Reason: Button is disabled (cart is empty)
└── Screenshot: failures/checkout-step3.png
3. Interactive debugging:
> "Pause test and let me inspect the page"
[PAUSED] Test paused at step 3
Browser window is visible for inspection.
Type 'continue' to resume or 'abort' to stop.
8. Extensions and Challenges
8.1 Beginner Extensions
- Add support for mobile viewport testing
- Implement cookie consent handling
- Create reusable test snippets library
8.2 Intermediate Extensions
- Build cross-browser testing (Firefox, Safari)
- Add video recording of test runs
- Implement parallel test execution
8.3 Advanced Extensions
- Machine learning for flaky test detection
- Auto-generate tests from user session recordings
- Self-healing selector system
9. Real-World Connections
9.1 Industry Tools
| Tool | Similar Pattern | Company |
|---|---|---|
| Percy | Visual regression | BrowserStack |
| Applitools | Visual AI testing | Applitools |
| Playwright | Browser automation | Microsoft |
| Cypress | E2E testing | Cypress.io |
| Axe | Accessibility testing | Deque |
9.2 Professional Applications
- CI/CD Integration: Run visual and accessibility tests on every PR
- Design Systems: Ensure component consistency across updates
- Compliance: Maintain WCAG compliance for legal requirements
- Regression Prevention: Catch unintended changes before production
10. Self-Assessment Checklist
Understanding Verification
- I understand Chrome DevTools Protocol basics
- I can explain visual regression testing workflow
- I know WCAG A, AA, AAA levels
- I understand AI-powered vs traditional assertions
- I can describe baseline management strategies
Implementation Verification
- Browser control works (navigate, click, type)
- Multi-step test flows execute correctly
- Visual comparison detects differences
- Accessibility audit finds issues
- AI assertions interpret page content
Integration Verification
- Full test suite runs on a real website
- Reports are generated with useful information
- Baseline management works correctly
- Tests are stable (not flaky)
11. Completion Criteria
Minimum Viable Completion
- Control Chrome through MCP
- Execute basic user flows
- Capture and compare screenshots
- Run simple accessibility checks
- Generate text-based reports
Full Completion
- AI-powered semantic assertions
- Full visual regression with diffs
- Comprehensive accessibility auditing
- HTML reports with screenshots
- Baseline management system
Excellence (Going Above and Beyond)
- Cross-browser support
- Video recording
- Self-healing selectors
- Parallel execution
- CI/CD integration templates
This guide was expanded from LEARN_KIRO_CLI_MASTERY.md. For the complete learning path, see the README.