GPT-5.6 Sol Sets New Coding Standard with 96.2% SWE-bench Verified Score

GPT-5.6 Sol Achieves 96.2% on SWE-bench Verified Coding Benchmark

For ML engineers wiring up autonomous coding agents, that single number shifts the baseline — repository-level patch generation is moving toward a one-shot problem rather than a careful review loop. The open-weight cohort, meanwhile, just got a fresh data point worth benchmarking against.

The closed-frontier jump

96.2% is the headline, but the practical takeaway sits in what SWE-bench Verified actually measures: real repository tasks resolved end-to-end against the project's own test suite. GPT-5.6 Sol's score, per BenchLM.ai's reporting, signals that the closed frontier has crossed a threshold where agents can ship diffs with minimal human babysitting. If you've been holding back on letting an agent auto-merge to staging or skip the second reviewer, this is the delta worth re-running your eval suite against — yesterday's gut feel is stale.

Where open models sit today

The same window brought a useful open counterweight. Kwaipilot released KAT-Coder-V2.5-Dev under Apache 2.0 — a 35B-parameter Mixture-of-Experts model with 3B parameters activated per token, built on Qwen3.6-35B-A3B, supervised fine-tuned on 127K examples and then RL-trained to clean up pathological tool behaviors like excessive parallel calls and content repetition. On SWE-bench Verified it lands at 69.40% — the highest score among open-source models at this scale, according to HackerNoon's coverage. It also posts 63.00% on SWE-bench Multilingual, 41.02% on Terminal-Bench 2.1, 93.43% on PinchBench, and 44.20% on Scicode, so the gap to closed frontier is wide on the headline benchmark but noticeably narrower on tool-use grounding. Text-only, Hugging Face Transformers / vLLM / SGLang / KTransformers compatible — dropping it into an existing agent scaffold is a sanity-check afternoon, not a research project.

What to verify in your pipeline

Before swapping weights, three quick checks we keep coming back to:

• Latency vs. budget — a 35B MoE with 3B active is friendlier than a dense 35B, but measure end-to-end agent loop time on your real repo, not synthetic prompts. The activation pattern matters under sustained tool churn.
• Tool-call discipline — the KAT-Coder RL pass explicitly targeted excessive parallel tool calls and designed Qwen3.6-specific reward penalties to suppress pathological tool behavior. Still, watch your traces for hallucinated tool names during the first 50 invocations — that's where regressions hide.
• License and provenance — Apache 2.0 covers commercial use, but if you're fine-tuning or distilling, document the base model (Qwen3.6-35B-A3B) for the compliance trail. Easy gotcha to skip until procurement asks.