Tsinghua Team's Agent Rewrites Its Own Setup to Cut Costs

Researchers from Tsinghua University and the AI company Z.AI report that their HarnessPro system, which lets coding agents rewrite their own setup, cut average cost per task by 26.1 percent against a prior method on three coding benchmarks, in results not yet independently verified. ^[1][6]

The saving is for the authors' Heavy version against a method called RHO, averaged across the three benchmarks, with GPT-5.5 running in Codex. Pass rates rose by an average of 2.0 percentage points. The paper is a preprint marked under review. ^[1][6]

What a harness is

An AI model on its own is a text engine. To work as a coding agent it needs scaffolding: instructions on how to approach a task, tools it can call, rules for when to check its work and when to stop. That scaffolding is the harness. ^[1][5]

HarnessPro leaves the model alone. Its weights, the numbers inside the model that hold what it learned, stay unchanged. What changes are the instructions, tools, skills, runtime rules and verification policies around it. ^[1][5]

The system works from the record. It reads logs of earlier agent attempts, looks for failures and for wasted effort, and hands the findings to specialized agents that plan and make the changes. Candidate harnesses are then scored on fixed tests written by a language model. ^[1][5]

The authors say this stage uses no ground-truth solutions, meaning no correct reference answers written by people, and no official tests or official grader feedback. ^[1][5] If that holds, the agent is improving against checks it wrote for itself, not against the exam it will later sit.

The project keeps three versions of the result. Lite aims at low cost, Medium tries to balance cost and performance, and Heavy aims at the highest pass rate. The chosen harnesses are frozen before the final run against the official benchmark tests. ^[1][5]

The numbers

The experiments cover three coding benchmarks: Terminal-Bench 2, SWE-Bench Pro and ProgramBench. The main model setup is GPT-5.5 in Codex. A second pairs GLM-5.2 with Claude Code. ^[1][5]

Both headline figures are averages of the per-benchmark differences, not one score pooled across all tasks. ^[1][5]

The Terminal-Bench 2 result is the one the authors lay out in full. Heavy passed 80.8 percent of tasks at a mean cost of $0.72 each, against 76.3 percent and $1.09 for RHO. The authors describe that as a 4.5-point gain and a 33.9 percent lower cost. ^[1][5]

By simple arithmetic, a 4.5-point gain on one benchmark inside a 2.0-point average across three leaves about 1.5 points of gain for the other two combined.

Cost here means billed tokens, the chunks of text a model reads and writes, priced differently for cached input, uncached input and output. It is not a full accounting of what running an agent costs a company. ^[1]

What the test can and cannot show

The samples are small. The project page lists 59 test tasks for Terminal-Bench 2 and 100 each for SWE-Bench Pro and ProgramBench, with 10 tasks per benchmark used for the evolution stage. Results are means over three runs on the first two benchmarks and two on ProgramBench. ^[5]

The scoring rules also differ. On Terminal-Bench 2 and SWE-Bench Pro, a task counts as passed only if every official test passes. ProgramBench reports the average share of test cases passed per task, which is a softer measure. ^[1][5]

A two-point average from two or three runs is the kind of margin that run-to-run variation can produce. What the evidence supports is a narrow statement: on these task sets, with this model and this tooling, the authors' pipeline came out cheaper than its predecessor.

It does not show that a company's own backlog of messy tickets will get 26 percent cheaper. It does not show the gains survive a new model version or a change in provider pricing.

The benchmark itself has a history of repair. The Terminal-Bench team says version 2.0 was manually and language-model-assisted verified after problems turned up in the first version, including tasks that broke when outside services changed. ^[3] That speaks to the care in building the test. It says nothing about HarnessPro's scores.

Open code, partial reproduction

The code is public under an MIT licence, so anyone may try to check the claim. ^[2] The evaluation guide is candid about the gaps. Benchmark data, task and grader images, model services and credentials are not in the bundle and must be obtained or configured separately. ^[4]

There is a sharper limit for the second model setup. The guide says the released GLM-5.2 harness snapshots are not included, so using the GPT-5.5 harness files will not reproduce the GLM-5.2 results. ^[4]

The paper lists Zhuoshi Pan and ZhiYuan Feng at Tsinghua, Yukuo Cen at Z.AI, and further authors at Shanghai Jiao Tong University, the University of Warwick and Kling AI. ^[1] It was first submitted to arXiv on October 19, 2026, and revised the next day. The current version runs 41 pages. ^[6]

The idea deserves attention. If an agent can learn from its own logs which steps were wasted, the saving comes from doing less, not from a bigger model. That is a cheap kind of progress, and the kind most likely to reach ordinary users.

It is also the kind that most needs a second pair of hands to confirm it.

What to watch is whether anyone outside the author list reruns the released GPT-5.5 harnesses on the public benchmarks, and whether the review process changes the paper's claims.