Untitled

You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each: 1. It overthinks — Just like the previous iteration. High confidence. 2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others. 3. It sometimes overthinks — Best case all around. A feature, not an impairment. One final thing worth mentioning: (I made myself irrationally angry writing this)

You might already know this, but a large part of test-time compute / 'overthinking' is just letting the model do more passes, and refine its activation residuals more. For example, even if you make thinking tokens literally just '....' (absolutely meaningless; zero information), you still see significant performance improvements: https://arxiv.org/abs/2404.15758 and https://arxiv.org/abs/2607.22925 for some starters. Treat thinking more like a "loading screen message" that's been RL'd to somewhat resemble its actual internal state; which happens in its activations, not tokens.

Did you observe the model overthinking on practical tasks? While 3.8 does think a lot on xhigh I've found that it really depends on the task. On one-shot prompts that are usually the first to be posted during new releases it will tend to spend a lot more time thinking than doing. In other words the more open ended a problem space becomes, the more Qwen will tend to second-guess itself. Conversely I've found that it can be as succinct as Muse Glimmer when it has a clear path forward. This can be either through well defined requirements or through unambiguous steps to take based on its own reasoning. While I do think it's fair to call out how much smaller model overthinks especially on one-shot prompts, in practice it hasn't led to an overall increase in time to task completion at least for what I've been using it for.

That's low reasoning for a model, but max for a HN comment.

You are absolutely right to push back on this. Let me think for a moment.