Verrrry thorough in its work and therefore verrrrry slow. It spent a long time with each question, running about 26 queries for each question, and in some questions it ran 64 queries. One of the queries that it wrote also took about an hour to run, which affected its overall exam time.
Last year the open weight models were the cheap & good alternatives. Then in September/October that cost story got rewritten by the GPT-6 series and Claude's 5 series.
Hy4 brings open models back on the cost/accuracy frontier, but its behavior is too thorough + slow.
Another thing we're seeing is that post training open models is starting to see results. Particle-1 from last week is also on our frontier, which is post-trained on DeepSeek V4.1 and Fireworks Ember-1 from a few weeks ago was post-trained on Kimi K3.
Take these numbers with a grain of salt: this exam is just for short horizon, low-context data analysis tasks, not full-fledged software projects.
> Looking ahead
> Although Hy4 is one of the strongest open-weight models available today, a notable gap remains between it and top proprietary models, particularly on challenging internal benchmarks for research-level mathematics, long-horizon coding, model self-improvement, and advanced visual question answering. We will continue to steadily scale pre-training and post-training to narrow this gap.
I like that they acknowledge that. For working with ai as a tool open models are already great. For millennium problems and doomermaxxing they are not there yet.
Slightly worse than Kimi 3.1M Pro (which just released open weights today) but much cheaper even excluding the current 50% off promotion, and Tencent promises to make it even cheaper once it goes open-weight. Interesting price economics.
Kimi 3.1 Pro is 2.8T vs this at 770B. I find that the jump in parameter count gives less tangible benefits like better "taste" and long context coherency, but it depends on the problem if that matters obviously.
increasingly feeling like the entire US economy hangs on whether or not Ant/OAI can stay ahead of the Chinese models.
If the gap closes, we do zero austerity, and interest rates are high, I cannot see a way out of a major economic crisis, even if we do start to see dramatic productivity wins.
More on-topic (and maybe more positive?), but this model feels a bit more expensive than I would have expected!
Videos 1 and 4 on vision and computer use are wild. They depict the model finding a 2m feature from a large satellite image, then determining where a photo was taken. Is it just me or do the Chinese model releases have less concern about looking like a military weapon? I don't think it's a bad thing necessarily, it seems honest. Also the music they choose for it is hilarious.
China hit maximum tech dystopia a long time ago. Now they can enjoy tech because they know its already maximally invasive and there is nothing they can do.
The price numbers look weirdly specific, I wonder where they came from.
Personally, the thing I'm most excited about is the FrontierCode 2.0 score, which is a benchmark I trust a bit more - 31 seems to be roughly on par with 6.1 Sol and Opus 5.5 medium
We've moved past per token cost a long time ago. Smaller models use more tokens. What's interesting is the cost per task, on that front deepseek isn't particularly good.
Landed on the cost/accuracy frontier in our data analytics benchmark: https://plotly.com/blog/tencent-hy4-plotly-data-analytics-be..., right in between GPT-6.1 Sol and GPT-6.1 Terra.
Verrrry thorough in its work and therefore verrrrry slow. It spent a long time with each question, running about 26 queries for each question, and in some questions it ran 64 queries. One of the queries that it wrote also took about an hour to run, which affected its overall exam time.
Last year the open weight models were the cheap & good alternatives. Then in September/October that cost story got rewritten by the GPT-6 series and Claude's 5 series.
Hy4 brings open models back on the cost/accuracy frontier, but its behavior is too thorough + slow.
Another thing we're seeing is that post training open models is starting to see results. Particle-1 from last week is also on our frontier, which is post-trained on DeepSeek V4.1 and Fireworks Ember-1 from a few weeks ago was post-trained on Kimi K3.
Take these numbers with a grain of salt: this exam is just for short horizon, low-context data analysis tasks, not full-fledged software projects.
> Looking ahead > Although Hy4 is one of the strongest open-weight models available today, a notable gap remains between it and top proprietary models, particularly on challenging internal benchmarks for research-level mathematics, long-horizon coding, model self-improvement, and advanced visual question answering. We will continue to steadily scale pre-training and post-training to narrow this gap.
I like that they acknowledge that. For working with ai as a tool open models are already great. For millennium problems and doomermaxxing they are not there yet.
Slightly worse than Kimi 3.1M Pro (which just released open weights today) but much cheaper even excluding the current 50% off promotion, and Tencent promises to make it even cheaper once it goes open-weight. Interesting price economics.
Kimi 3.1 Pro is 2.8T vs this at 770B. I find that the jump in parameter count gives less tangible benefits like better "taste" and long context coherency, but it depends on the problem if that matters obviously.
increasingly feeling like the entire US economy hangs on whether or not Ant/OAI can stay ahead of the Chinese models.
If the gap closes, we do zero austerity, and interest rates are high, I cannot see a way out of a major economic crisis, even if we do start to see dramatic productivity wins.
More on-topic (and maybe more positive?), but this model feels a bit more expensive than I would have expected!
Videos 1 and 4 on vision and computer use are wild. They depict the model finding a 2m feature from a large satellite image, then determining where a photo was taken. Is it just me or do the Chinese model releases have less concern about looking like a military weapon? I don't think it's a bad thing necessarily, it seems honest. Also the music they choose for it is hilarious.
China hit maximum tech dystopia a long time ago. Now they can enjoy tech because they know its already maximally invasive and there is nothing they can do.
That makes a lot of sense.
The price numbers look weirdly specific, I wonder where they came from.
Personally, the thing I'm most excited about is the FrontierCode 2.0 score, which is a benchmark I trust a bit more - 31 seems to be roughly on par with 6.1 Sol and Opus 5.5 medium
How does it do with tasks anthropic doesn’t like? This is my main use case for open models.
Not cost competitive. The performance difference is not commensurate with the price.
We've moved past per token cost a long time ago. Smaller models use more tokens. What's interesting is the cost per task, on that front deepseek isn't particularly good.
https://artificialanalysis.ai/#intelligence-efficiency-tabs
Wild that this is the first question I ask at this point. After that it’s a question of how it stacks intelligence wise vs glm or ds.
OAI / Ant are in for a reckoning.