Benchmark models by building one feature in your repo

Goal: Pick the right model for my real work, not public benchmarks.
Setup: agents used in source: Claude, Codex; tools/MCPs stated: mutation testing.
Steps:
1. Pick a real mid-size slice of your project and write the contract.
2. Have each model build it in isolated environments (Claude Code and Codex).
3. Before looking at any build, write hidden tests from the contract alone (47 here).
4. Mutate the contract (4 mutations) and run each model’s own tests against them.
5. Check behaviour on the real binary by hand; compare cost and wall time.
6. Review the contract before dispatching: that’s where the expensive model earned its keep.
Rules: ask me before anything irreversible (sending, posting, paying, deleting); don’t invent data, results or tools I haven’t given you; tell me what you couldn’t verify.
Based on: Damian Pala — https://x.com/iamdamianpala/status/2098086060081221977

Steps are from the source. Includes a “Based on” credit line.