---
name: harness-effect
description: Compare two AI agent harnesses across task success, latency, and run cost to isolate the effect of prompts, tools, memory, retries, and orchestration. Use for agent runtime evaluations and harness selection.
---

# Harness Effect

## Definition

The harness effect is the measurable change in an AI agent’s quality, token use, latency, and cost caused by the orchestration layer around its model. Tool selection, prompts, memory, retries, and control flow can make the same underlying model behave like a meaningfully different system.

Reviewed 26 July 2026.

Compare agent setups while keeping the underlying model and evaluation set fixed.

## Workflow

1. Confirm both harnesses use the same model, tasks, scoring rubric, and sampling settings.
2. Collect success rate, median latency, and average cost per run for each harness.
3. Weight task success most heavily; treat latency and cost as explicit penalties.
4. Calculate the score difference and surface the raw metrics alongside it.
5. Identify which harness component likely explains the change.
6. Recommend an ablation test before attributing the entire difference to one feature.

## Output

Return the winning harness, score margin, metric table, likely driver, and next ablation.

Flag comparisons with mismatched models, tasks, or sample sizes as inconclusive.

## Research basis

[The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI](https://arxiv.org/abs/2607.06906) — Ali et al., arXiv (2026). The paper introduces the exact term and measures orchestration’s effects on quality, tokens, latency, and cost.