# Counterfactual Test

Use this when you need a credible comparison between AI-assisted work and non-AI work.

## 1. Work Unit

Choose a task with enough volume and clear acceptance criteria.

Examples:

- Prepare a customer meeting
- Review a contract clause
- Write a proposal section
- Summarize customer notes
- Answer HR or IT questions
- Triage support tickets
- Classify incoming requests

Work unit:

Acceptance criteria:

Minimum sample size:

## 2. Comparison Design

Pick one design.

- Holdout team
- Random task split
- Matched historical sample
- Repeated task design
- Expert-reviewed sample

Chosen design:

Reason this is fair:

## 3. Measures

Primary metric:

Baseline value:

AI-assisted value:

Guardrails:

- Quality:
- Review burden:
- Risk:
- Employee effort:
- Customer impact:

## 4. Full Cost Capture

Include all AI-related work.

- Prompt/setup time:
- AI wait time:
- Review time:
- Correction time:
- Escalations:
- Training/support:
- License/runtime cost:

## 5. Result

Sample size:

Median change:

p90 change:

Quality change:

Cost per accepted output:

Confidence level:

## 6. Decision

Choose one.

- Scale
- Change
- Retest
- Stop

Decision:

Reason: