A Better Harness Can Unlock Smaller Models

Agent performance is often attributed primarily to the underlying model. In practice, the same model can behave very differently depending on the tools, context management, execution loop, and delivery checks around it. That surrounding layer is the harness. A low benchmark score can reflect either a capability gap in the model or a failure in the agent design.

We saw this with Qwen3.6-27B on the Harvey Legal Agent Benchmark (LAB). Each LAB task gives an agent a partner-style instruction and a set of use case documents in a sandbox, then scores the resulting deliverable criterion by criterion. Across 140 held-out tasks, changing the harness moved the base model's criterion pass rate from 67.10% to 85.92%. Post-training of the model raised it further to 88.03% bringing it close to frontier performance.

The gains came from two sources. The harness recovered capability already present in the base model, while post-training added capability it didn’t have at all.

Trajectory analysis showed that much of Qwen's lost score came from failures in execution and delivery, before the quality of the legal answer could be evaluated. Some runs ended without a usable file. Others looped on malformed tool calls or lost track of instructions after large file reads.

We used training-set trajectories to identify recurring patterns, optimized the harness based on the patterns, then evaluated the changes on 140 held-out tasks with no overlap. The optimized harness bounded file reads, returned actionable tool errors, preserved key state during context compaction, and validated deliverables before exit. Missing-path write failures fell from 120 to zero.

Qwen's weights were unchanged. The improvement came from reducing losses during execution and delivery. The experiment then paired that harness with post-training, which let us separate capability the harness recovered from capability the model lacked.

Figure 1. Criterion pass rate across 140 held-out Harvey Legal Agent tasks.

The effect was not limited to Qwen. It also extended across model sizes, including larger, frontier-class models. Applying the same harness to GLM-5.2 moved its criterion pass rate from 88.39% to 90.64%. Smaller evaluations also showed gains with Gemini 3.6 Flash and Claude Opus 4.8, although those results should be treated as directional because they used narrower task subsets.

We compared our optimized harness with Prime Agent 0.7.1 and OpenCode 1.18.23 on a smaller task subset while holding the model fixed.

With Qwen, our optimized harness reached 84.13%, within 0.79 percentage points of Prime Agent, while using 54% fewer input tokens (8.06 million versus 17.48 million). OpenCode used fewer tokens at 6.38 million, but scored 75.40%.

With GLM, our optimized harness reached 92.06%, beating Prime Agent by 3.96 percentage points while using 18% fewer input tokens, and OpenCode by 5.16 points.

Figure 2. Performance versus total input tokens on the smaller harness-comparison subset.

Token volume alone did not explain performance. Harness design changed how productively each model used its context.

That translates directly into cost. Achieving better accuracy with fewer tokens lowers inference cost. Running an optimized smaller model in place of a frontier model can be over 10x cheaper.

Once the major execution failures were removed, the remaining errors were more likely to reflect real limits in what the model could do. We trained Qwen on successful end-to-end trajectories containing the full sequence of reading, tool use, analysis, repair, and delivery.

With the optimized harness, post-training moved the criterion pass rate from 85.92% to 88.03%. The gain was smaller than the harness gain, but the share of tasks that passed every criterion increased from 5.67% to 9.22%.

Optimizing execution first also improved the value of the post-training data. A reliable harness produces cleaner evidence about what the model still cannot do, and better trajectories to learn from. Post-training then changes the model's behavior and may change what the harness should optimize next.

A larger model may still be the right choice when the remaining errors reflect capability limits. Choosing well requires separating those errors from failures in context management, tool use, or delivery.

These results point to a broader definition of AI efficiency: useful work completed for a given inference budget by the full agent system, model, and harness together. The gains come from co-optimizing both rather than defaulting to a bigger model or a better harness in isolation.