Easily the most interesting part of this announcement is buried in the second to last paragraph:
"We're also launching GPTā5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity."
750 tokens/s on a frontier model is going to be extremely interesting. I doubt this new version is anything but a version bump in terms of capabilities but if we can start getting these answers back faster, they end up being more useful.
Just off the top of my head, I can think of the tedious task of finding certain functionality within a codebase. I usually can't beat an AI agent harness at this task today. If the AI model is 3x faster I have less of chance.
Thatās an awful visualization. I can skim code quite quickly, but not when it shows up one character at a time in a small window, modem style.
At least that site should draw out a full page then start replacing that page with the next, starting from the top and working downwards, repeating each time it hits the bottom.
Not for prefill. I suppose if you just want to imagine what generation speed looks like in the current generation of TUIs, itās an okay visualization.
I'm not sure if that's what you were going for, but I read it as if it were written by The Board in the game Control, and found myself with the appropriate level of existential dread.
Because there's been nothing to discuss since their announcement. Their API access immediately closed due to overwhelming demand and they didn't fab newer models than Llama3 yet.
Probably they will make bank selling to HFT for a while.
Funnily enough, pasting your comment straight into Jimmy leads to a... Funnily suboptimal answer that does not answer the question.
As someone else already contributed, this is driven by a Canadian startup taalas that basically makes chips that are llms, so everything is very fast but also, baked into the chip. Once this kind of stuff is a commodity in like 10 years, our world will be very, very different.
Taalas HC1 AI uses Llama 3.1 8B, but takes up a massive 53B transistors and 815mm2 on TSMC N6 (nearly at the reticle limit of 858mm2). N2 is a little less than 3x as dense (110MTr/mm2 vs 313MTr/mm2).
This chip would still be 272mm2 on N2 which is an eye-watering $30k/wafer and bigger than a 9950x or Nvidia 5070.
This just isn't feasible. Some of the latest-gen LLMs seem to have 5-10T parameters or about 1000x more. I don't know that taping out just one chip makes economic sense let alone the 300-1000 chips required for a cutting-edge model. Things like continuing education so your model knows about the latest NPM packages or world news is super important, but seems like it would require new chips.
There are a TON of uses for an 8B parameter models on the edge, but this is WAY too big to put on the edge of anything. Something like a 10mm2 100m parameter voice model might be feasible on the edge, but only for expensive devices, but most of those are TSMC 28nm (up to 29MTr/mm2) or GF FDX22 (up to 40MTR/mm2) which would increase the AI chip to the point where it would absolutely dominate the BOM.
> Things like continuing education so your model knows about the latest NPM packages or world news is super important, but seems like it would require new chips.
They probably have a few ideas around that. Me, personally, I'd have one main expensive chip (replaced every 10 years, or whatever), with a secondary cheap chip in front of it that gets replaced every year or so.
The secondary chip could act the way RAG does, or perhaps both chips together can act as LoRA.
Either way, 99.999% of the knowledge is static, you just need to fine-tune the weights with that remaining 0.001% knowledge, which can be done using RAG or LoRA on a much smaller (thus cheaper) disposable chip.
Yeah, they're clearly just starting out and just shipped their very first proof of concept. But to me, their plans seem generally reasonable https://taalas.com/the-path-to-ubiquitous-ai/, and like I wrote, if this kind of thing succeeds and could become some kind of cheaply producible commodity component, I think there's huge value in that. Alas, maybe not as a frontier model replacement, but say 10 years from now you can drop a cheap raspberry pi like device in your Lan and have a fast local engine for things like text sentiment analysis, text summarisation, voice recognition, basic vision and things like that, that would be pretty exciting to me (but maybe as you outlined, impossible in practice)
Sometimes I visualize a setup like this [0], based on 2D art by Simon StƄlenhag. Someone has their home robot sitting on a desk connected to their old PC with thick cabling, dumping endless lines of each subsystem's <think> logs to diagnosis why it did something weird earlier in the day. Systems pushing 750+ tokens per second per subsystem might even be considered on the slow side for realtime tasks by then.
What about 15k tokens per second? [0] I remember looking at this earlier in the year and it being so fast that it feels fake. And, yes, this model is old - but still awesome for what it is.
Itās not just old, itās also tiny and quantized. Itās llama 3.1 8b at 3/6-bit quant. This is the type of thing you can run on almost any deviceā¦
But itās irrelevant. 750 tokens/s on a full frontier model is useful. 15000 poor quality tokens is much less useful no matter how much scaffolding you put around it.
You are missing the point. This is a technology demonstration on prototype hardware, and no one intends it to be seriously useful.
Their architecture has fundamental speed and efficiency advantages over GPUs or Cerebras. They expect to scale up to real LLMs by splitting a model layer-wise across several chips, which they can do without incurring any throughput penalty.
> They expect to scale up to real LLMs by splitting a model layer-wise across several chips, which they can do without incurring any throughput penalty.
Iāll patiently wait to see this in reality. Their demonstration hardware is a 250W chip that is enormous in die area for the model size. Theyāre making a lot of claims, but until they can deliver then itās nearly vaporware in my view.
Iād be happy to be proven wrong, but I think theyāre going to quickly run into hardware realities quite soon if they think they can just chain a bunch of chips together to achieve the same performance on larger sizes.
Why can't they do it? Jim Keller's company is also taking a different approach [0].
The simple fact that we think what we have now is scalable is basically what you are saying can't be done: " just chain a bunch of chips together to achieve the same performance on larger sizes". How do you think current architectures work? And what is being used today is all proprietary to one company!
Actually it's the opposite. Per mm of silicon it's massively less efficient and making enough chips and powering them is a major bottleneck right now. Worse, scaling to larger models requires more of our absolute best quality silicon manufacturing, where e.g. an H200 mostly just needs more memory.
But Iām not missing the point. If you can run one frontier model at 750t/s, then you can probably run many many instances of an SLM in parallel at a rate that exceeds 15k/s. Thatās kinda the point of the flash or ultrafast variants. And theyāre on something much more modern than llama3.1.
Yes, you are missing the point. 1) It's a demo. [0] 2) It hasn't been updated for 4+ months.
You don't need LLMs for everything. That is 100% the point. You can burn down the world with all of your frontier LLMs that are being used for simple queries OR we can do something faster and more efficient like this. Just because you can run a SotA model at "fast" speeds, again, severely misses the point.
And no, you can't run anything from Anthropic or OAI on-prem, so until you can there's really no comparison. If people want to continue down the path of gate-kept models with no other options then we'll all follow you off the cliff.
Why are you representing this as such a binary here? For SLM we donāt need the Taalas stuff at all. Just run it locally on your own device if itās truly a small model. And thereās plenty of larger models that can be run on-premise just fine.
I think itās impressive that a frontier model can achieve 750t/s. Thatās all. You can get similar insane token speeds from other open weight models too.
The irony here is, according to you, my take is the binary one. When your response is: well, we can all just run it on our devices - we don't need any other options!
You seem to be cool with a very small and gated ecosystem with whatever tech billionaires want you to have access to.
I grew up in the era where compute was diverse and open. You may think this is OK, but it's not. The more options we have and the more diversified they are the better tech will move back towards.
I'm not the one with the myopic view here. Enjoy your "on-device" models over in your utopia of a walled garden.
I think youāve got things quite backwards if you think that the desire to run models on device or use any of the variety of open weight models (big or small) on premise is somehow bowing down to tech billionaires. Quite the opposite really.
Once again, my statement is that the Taalas product is not a fair comparison because it runs an old outdated model. If you want to run a similar model at similar speeds (albeit not serially, but in parallel) you donāt need their product.
> Once again, my statement is that the Taalas product is not a fair comparison because it runs an old outdated model.
Either you didn't look at the page I linked or you're having comprehension problems.
> If you want to run a similar model at similar speeds (albeit not serially, but in parallel) you donāt need their product.
Except, you can't. There's no commodity hardware out there today that can run even an "old outdated model" at this speed and power utilization. Again, maybe read first and try to understand my original point?
> "...my statement is that the Taalas product is not a fair comparison..."
You actually hadn't stated this. You said it wasn't needed. Which is it?
> If you want to run a similar model at similar speeds...
You can't. Find me a single system that can run this, again, "old outdated model" at even similar speed. You're hung up on the model. The point is that if we all just stay in this wonderful world of inefficient large models we will all end up at the mercy of OAI, Anthropic, Google, etc. When other companies, like Taalas are putting research dollars in to making AI scalable, affordable and efficient. Do you really think commodity hardware is going to be attainable anytime in the near future on this trajectory? Do you need a laptop to cost $10k USD before it clicks? That is exactly how you end up kissing Altman's ass in this situation.
Yeah, that's the point, right? With tool calling the LLM becomes code. So instead of asking it to write an accounting software, you can hire the LLM to be your accountant.
Not necessarily. Consider a human assistant who performs repetitive tasks at an acceptable cost and accuracy while dealing with edge cases often autonomously.
If we want reliability - we come up with processes to make it reliable and not rely on individuals getting it right. Code is a way to create a reliable process in the digital world.
For some things that's acceptable or even good. If I want to add up a list of a million numbers human assistants aren't bringing any advantages though.
Using gpt-5.4-mini in off-peak hours already feels like super-speed to me. That's probably no more than 100-150 tk/s. I can't imagine 750!
I've always eyed Cerebras but never had a use for it that would justify paying for the API directly. Although now that I think about it, trying out the API would probably cost less than a subscription for a month...
Agreed, 1000tok/s just fills up the context window (which is big by 2004 standards) super fast. But seems like 5.3-spark was just a taste of whatās to come.
The ChatGPT subscription gives you access to the -spark model(s) in Codex which are blazing fast (but pretty dumb) which I think runs on Cerebras hardware too.
is this specifically in codex? have been trying to use the models for months on opencode then pi but it says chatgpt subscriptions don't have access to it - i was under the assumption that OpenAI doesn't lock down their models based on harness a la Claude Code
I have a pretty good use case for gpt-oss. The amount of time savings has actually been wild. Definitely worth a try. Just to be clear, it gets like 2000tok/s
But it seems that there is some queuing/load balancing on their side, I mean when opus is actually outputting this 55t/s it feles fast, but apart from it's internal reasoning I think there's sometimes just waiting.
Oh wait yeah good point. At 750 tokens a second and the same amount of human patients they can set it to think for the same amount of time but four or five times the amount of thinking tokens, which may improve the quality of the eventual output.
the more advanced models also utilize a lot more tokens, and a lot of these extra tokens may go towards safeguards at a higher rate than prior models as well.
not to say a speed boost isnt there but if they didnt increase tokens / s at all youd likely see things slow down a lot with the new model compared to current
> I can think of the tedious task of finding certain functionality within a codebase. I usually can't beat an AI agent harness at this task today.
Yup, I remember "racing" the AIs to figure things out in codebases just a year ago. Today, I have no chance. Whether it is due to degraded reasoning capabilities on my part or better models, I don't know.
At least in my case, much of the code in the codebase I'm working on is AI generated so even if I have an accurate mental model of how everything works, I have no idea where any of it is located or named.
I seem to remember - but cannot find, even with an AI boost - someone's "law of computing" or somesuch describing the amount of time that has to pass before code you wrote is indistinguishable to you from code written by someone else. At any rate the interval is not so long.
In your own codebase you can at least run a "you simulator" to arrive at the answer fairly reliably. "Where would I have put this bit? Oh yes, of course ..."
I'm skeptical of how fast "up to" 750t/s really means. Maybe if they make it extremely expensive so it frees up enough capacity?
GPTā5.3āCodexāSpark currently runs on Cerebras chips and it's giving me around 150t/s. Still relatively very fast, but nowhere near the 1,000t/s they claimed at launch. (Also it's not a very good model.)
That said, I'm super bought in to faster models being better for most use cases than smarter models.
Yep this is a glimpse into the future of 500+ t/s, which is in my opinion the next big thing that validates Jevon's paradox (the models are already smart enough)
Yeah! So at a much smaller scale, being able to boost Step 3.7 Flash up to 40tk/s on my Spark-alike with proper triple head MTP was the thing that made it superior to Qwen 3.6 27B in wall clock time despite Step reasoning more
A lot of the open Chinese models get their results through huge reasoning loops. Being able to boost decode perf is what will make them worth it, and Iām sure OpenAI and Anthropic could do similar (if they arenāt already)
āSmart enoughā really depends on how many other people have encountered a problem close enough to yours and solved it somewhere on the open internet, IMO.
Most of the frontier models can, when prompted and tooled correctly, do a lot of āreasoningā tasks that amount to resolving how the user has explained a particular widely known paradigm.
The more difficult and obscure the issues you provide them with, the faster you notice them reward hacking by altering the criteria until they are no longer attempting to solve the problem. Using āadvisorā style loops helps hold this off at the cost of tokens, but there is still a fairly short limit at which they will essentially give up if they canāt find all of the necessary information - sometimes the issue is actually worse if they find a small amount of information instead of nothing - theyāll extrapolate from that tiny piece of data and generate plausible-sounding hallucinations almost every time.
And god forbid your problem involves doing something a different way than the majority of people do it. Unless you can write a full spec on it, the models will repeatedly spiral back into adjusting everything about your problem until it matches one of the most popular approaches in their training data.
I get how this is a trueism now but I never really understood why it would be useful to scrape cc/codex sessions for training. The relative amount of human input for that is so low (isn't that why they are so loved and used?), how could it actually be useful to them? Wouldn't you wanna focus on people not using it?
It's more useful as a set of feedback on the model results. You can do sentiment analysis on the user responses to see if they found the model results useful/frustrating/etc and use that to guide future training
I think this is a rosy estimate. The vast majority of what people do with these models is just the same old shit, I would be surprised if 1% of it were genuinely novel stuff worth folding back into the training data.
I think the glimpse that is there will be exclusive access. So much for the open in openAI. If this technology really transforms society in the ways expected with inequality an unavoidable consequence equal access should be required like internet access was (isp canāt give preference to specific user traffic)
At a certain rate we will be able to move towards continuous / real-time inference systems. The discrete, turn based solutions are quite confining with how they must be trained. Continuous and real-time would fundamentally alter the domain.
From an information theory perspective we are still in dial-up territory with regard to the actual information rate. 750 tokens per second would be a really bad dialup connection. Imagine 10 millions tokens per second.
We still have the problem that auto regressive decoders are memory bound.
The new Blackwell hardware combined with TensorRT-LLM and speculative decoding consistently can hit 1,000 TPS/user barrier, comparing to closer to ~250 TPS/user (out of 10k+/TPS on the server)
Is there something I missed, this looks more like 14.4 to 56 on a 64kbps backing channel modem story. I have no doubt that there are still massive gains to be found, but they seem to be using existing constraints more efficiently, not that fios is coming.
I donāt have the budget to work on the foundational model scale, but with a draft model 10xā20x faster than target and an 60-80 acceptance rate I can see how they could promise 750/TPS (with a lot of other hard work) but I would appreciate where I should look to figure out what I am missing.
Is there anyone exploring or writing about this in public? I've felt for a while that the turn-based model was not quite right, but also felt too stupid and ill-informed to have much of an opinion about what else it could be.
Thinking Machines, the started founded by former OpenAI CTO Mira Murati. The interaction models demoās in their videos imo breaks the awkward turn-based barrier. Returning responses quickly reaches a threshold where it starts to feel like a natural conversation. Their approach to solving this problem is rather clever.
I have an active 'sleep' mode, where when the user is AFK the LLM goes into a loop with a sleep 10 between turns, and determines (via tool use) if something should be done. That's still a 'turn' in a way, but it's all the LLM just sort of sitting around like a human would, pondering what to do next.
But I could imagine after each space(eg, word) having a 27b model on a nice rig, with thinking off, doing a quick look at the sentence and determine if it should interrupt and start a real turn with thinking on. Which kind of is non-turn based in a way. If you're typing fast, it might hit that run every 3 or 4 words, but that's sort of how a human might be when a person is talking to them. That is, waiting for enough info to interrupt, if needed.
There might be a way to process chunks of a sentence using commas as break points, eg for comma delimitated phrases in sentences, so the whole sentence doesn't need to be re-processed each "should I break in" assessment at word break.
Could be fascinating. Could actually do some of this right now.
I don't think this is what the parent poster was thinking, but the idea even at this level seems fun.
Yeah, I've played with some similar stuff on my 9070xt. But ultimately all the ceremony on top is cloaking that it's still just two or more models taking turns prompting each other to give the illusion of continuous thought. It's still one thought at a time, with every thought starting from scratch with a big chunk of prior context.
The idea of true continuous thought and memory-generation is very interesting, though I can't even begin to conceive of how it would work.
Or if it's even correct? Maybe our brains are secretly actually turn based too?
I think they're definitely attention based. They're just immensely faster than LLMs, because a lot of processing is in silicon in a sense. Think of a ball flying towards you, you don't have to think, the data is handed to your conscious mind, speed, direction, which literally knows how to snag the ball out of the air.
But we have multiple things vying for attention, and some are immediate. Being on the phone talking to someone with great attention, and then touching a burning surface -- you immediately pull your hand back (lizard brain) before even being aware you're doing it. The same with peripheral vision and something surprising coming at you from the side. It snags your attention.
So maybe we are turn-ish based, but just multiple parallel processes each with their own turn? Neurons have their own 'trigger', and I think the brain has layers of triggers, each aggregating and filtering up to the top which then triggers.
I think doing this all with an LLM is silly, some of it should be innate, such as peripheral vision. Data handed to the main thread when triggers occur. I wouldn't want an LLM to handle "walking" fully either.
Some octupus have a sub-brain in each tentacle, each thinking and feeling, there are serious questions as to what its mind is like. I feel initial LLM powered androids may have to be like this a bit.
I agree with you however I think even then you're still giving our brains too much credit. The speed definitely comes from that processing being "in silicon".
Your ball throwing example however will be handled by really small and really fast "fine tuned agents" dedicated to catching that ball. Eyes to motor neuron system. There are the illusion of free will experiments that demonstrate your brain only rationalises and explains whatever activity took place after the fact (It's explanation may even be entirely wrong).
And more importantly those 10 million tokens/s should cost fractions of a penny. Tokens need to be dirt cheap so I hope they build out massive solar+battery powered data centers asap.
No anything but wasteful, weak, expensive, environmentally harmful solar. Nuclear is the only path forward for superior energy production, at least until we figure out fusion.
Your comment made me think of another real time. Real time, dynamic code/apis.
Imagine a world where there is no code, just things mildly handshaking and then creating data APIs on the fly. Where communication is fuzzy and locked in on an individual basis. No years of RFCs, no RFCs at all, just... data.
Just data, man.
An API arbitration aberratically assigned at authorized access, abridged and annotated, analytically assuring absolute assurance.
Why remove the code and binary artifacts, though? Don't you want to verify that the business logic is accurate and the processing is deterministic?
In some circumstances there is no substitute for something that you know will produce the same answer for a given input, consistently. And that's before even considering the watts per response.
The AI is the business logic, and the processing, and all of it. The context window is effectively infinite, with layered context window depth and speed.
Think of short and long term memory, or think of RAM vs SWAP. Dip into swap to pull needed data into RAM context. SWAP can be anything storage related, including a symbolic database or a best-encoded set of priorities.
If a person knows 100 knots, but hasn't tied one in 23 years, they might have to think a bit before they get full use of their long term memory... and tie that knot. I don't see an issue with layered speed context, that is, GPU ram, slower RAM, DB storage, all in the same format.
Imagine a world where a 'factory' is just high-tech 3d printing, with a dozen different methods (eg, plastic, laser+metal, etc), and getting specs for everything possible is, well, an immense amount of work. Imagine having a billion item catalog of things to print, and, imagine new requests for new things to print.
And the request doesn't come from an expert, but from some dude who sketched something on the back of a cardboard box.
The LLM can pull from long term storage for how those things were done before, how similar things were done before, and just get to work.
Regardless, the connection was what I was talking about before. Data transfer. Do you need http? json once established? What? Imagine instead that's all in the wind?
And it's so fast, so capable, that dynamic is easy.
It's very easy to see how world changing this technology will be. In a few years these AIs are going to be negotiating how they communicate with each other. Humans won't necessarily be included in that negotiation unless we have some kind of specific reason to. So many communication layers are going to be opaque to humans. We just have to trust our AIs are communicating efficiently and safely.
What this made me think of is life before computers, where people mildly handshake, create agreements on the fly. "Where communication is fuzzy and locked in on an individual basis."
TBH, to me, this imagined future looks a lot like it'd have all the problems we already have.
Neat. Not precisely what I was thinking, but 100% definitely very cool and the same mental scope. It's like we wear different shoes, but go to the same cobbler.
I can imagine shoe-horning* this so the agent saves prior builds of every successfully delivered or deployed item. In my example, perhaps if someone orders new design $x, it's shipped, and review is 4+ stars, it gets added as 'successful builds'.
* have to keep with the shoe theme, even though shoe-horning is not really necessary
Iāve thought about this before. No flaky config files, no updating endpoints, no status monitors. Just fuzzy everything that works almost all of the time.
Itās awesome, particularly since itās at DeepSeek tier prices (3X of DS-V4-Pro). At 1,000 tok/sec though you can really rip through tokens. (About $9 an hour if you manage to run the output nonstop.)
It tends to cost more than DS since it doesnāt seem to have as many input cache hits.
bean in mind that "GPTā5.6 Sol on Cerebras at up to 750 tokens per second" not necessarily means the same model (in terms of inference result). It can mean anything like a very quantized model, a different level of model activation per inference etc.
Of course we can trust that wouldn't name the same thing with different levels of intelligence, right? Right?
I still use GPT-5.3-codex-spark which also runs on the Cerebras chips. Spark can run at >1000 tok/s but it's highly limited in it's context window size so it's not suitable many workflows.
Granted this will be a bit slower (relatively speaking) but it will still be awesome.
Same - I had some "AI-assisted coding interviews" where I had to bring my own AI tools, and found the speed of codex-spark to be important for making progress quickly (and not sitting there waiting on Opus to think for 10 minutes).
There's a word for this that you should never pass up an opportunity to use: penultimate. (You should also never pass up the opportunity to use "defenestrate," but it sadly does not apply here.)
A friend of mine had his visa accepted because of this. He was explaining what he plans to do in US and he threw in āpenultimateā into a sentence somewhere.
The council stopped him, said that if he knows such words he definitely wonāt overstay his visit to work as a dishwasher, and accepted his B1/B2. Seriously.
Not sure if it would be the same if he used ādefenestrateā when talking about his plans.
This is a strange one. We know the hardware capabilities of Cerebras force them to do aggressive REAP pruning to serve Kimi K2.6. Meaning that about 750B parameters is the upper limit of what they can serve economically. Not sure if this means Sol is smaller than anyone thinks or that they're just going to charge so much that a very inefficient serving regime is feasible.
Last I heard, Cerebras chips were entire wafers and would be extremely expensive. How could OpenAI possibly have enough of these to serve a popular model at scale?
Cerebras is Milli Vanilli. They spend 10 years burning cash on a failed idea (which is frankly insane, since they should have figured out the limitations of heir stack in like... a weekend) and struck accidental gold with their 'Giant ass wafer'.
The company is valued like they broke open the grail, when in reality it's more like they bought a Cybertruck, got it stuck in the mud, and realized "You know what this thing does better than all other cars... shovel mud"
I'm shorting Cerebras with margin to virtually zero.
It all depends on the context window size. A small context size with fast performance won't be very useful today, as most workloads (like requests behind codex) usually have very long context.
This would be amazing for some of our "real-time" workflows, that need to fallback to AI for one reason or another. What used to happen is a rules based system did the majority of work, and occasional corner case would fall back to humans. Then we moved AI in, still not real time, but much faster. Cerebras could make that even faster.
OpenAI also announced two days ago that they're starting to make Cerebras style chips themselves [0], will be interesting to see how fast SotA model inference will be by the end of the year.
I don't understand how you refer to this as "Cerebras-style". Cerebras is wafer-scale and unique. Jalapeno is an inference-optimized conventional chip.
Even if their chip is a difference maker, end of the year is wayy too optimistic. Itāll at minimum be a multi-year effort to bring it to production at scale.
I don't see any indications that OpenAI is doing wafer-scale work.
I tend to doubt they would. Cerebras notably doesn't have a kv, is wildly high bandwidth, but within/across the chip, not able to dump/restore kv super well. I doubt openai is going to build something that is as expensive to run. Also, wafer-scale is absurdly hard & weird to pull off, so I doubt that would be their first foray.
At thousands of tokens per second, LLMs (harnesses) can start to do a broader tree search of possibilities even in inefficient token space. This unlocks capabilities outside programming.
Does the Cerebras variant offer input caching and corresponding discounts? Last I checked Cerebras would not cache or would cache but not give discounts for the cached input, making it impractical for agentic use and multiturn conversations.
"we can start getting these answers back faster, they end up being more useful."
Dude, 10x token speed is going to be absolutely nuts. Half the "parallel subagent workflow" business seems to be driven simply as a means to avoid tapping your thumbs waiting for the infernal robot to finish something. If things come back speedy quick all the time, it should keep up with the "speed of the human" and let me stay focused on one thread instead of half a dozen. Plus the cost of screwing up gets significantly lower because you just re-fire with an adjusted prompt and iterate.
Someday these things will be 100x as fast as they are today and that is when things will get insane.
it also makes the parent brain-dead because all those subtokens are missing from the context thus unable to steer the hyper dimensional context driven generation, and the subagent is dumb as a post so synthesizes something very weedsy while you're specifically attempting to understand the forest
You have an agent spawn the agents for you! You can ask Claude to do it for you, he is happy to use sonnet when you ask for grok and opus high when you ask for deepseek.
> I usually can't beat an AI agent harness at this task today. If the AI model is 3x faster I have less of chance.
Yes: we have these new tools that are extremely good at helping us search through our codebases. Not just to find where/how functionalities are implemented: IMO bug searching is even way more powerful.
But: why would you want to compete with AI to do that? I cannot compete with grep/ripgrep... And I'm cool with that.
This lets you focus more on the more interesting parts, where AI/LLMs suck fat balls.
From what I know about batch processing/ concurrency in inference this is a pipe dream... Or its going to cost an arm and a leg. I think they're lying or its going to be a much smaller model and not "frontier"
You have speculative decoding that easily increases speed 2-4 times with no loss of quality, and of course MoA architectures that speed up inference 10 times or more, although with some quality loss.
Better hardware, and other techniques on top of that and you speed up even further.
If you have no need for Anthropic/OpenAI's frontier model capability, you may be better served with an open-weight model that can't be taken away.
Edit:
> GPT-5 does the job.
I bring up DeepSeek V4 Flash a lot on HN, but I want to mention that according to Artificial Analysis, it trades blows with GPT-5 (high) (from August, 2025) [0]
Though with open models you have a lot of choice where to get it from. I see like ~15 providers here with various logging/ZDR policies, so pick whatever mix of price to features you want:
I don't believe a single word from AI companies, no matter where they are from. Sourcing their training data is run like genuine criminal enterprises - last year Anthropic settled for 1.5 billion, and and if they settled so quickly it might mean what we would see in court is even worse.
When I use their API I use it knowing that they probably train on the data, and knowing that it's probably used to improve future iterations of their models.
But I use their API extremely rarely lately, because local Flash is good enough for me the vast majority of the time
DeepSeek V4 Pro is only ~3-4x as expensive as Flash. It won't replace GPT-5.5 (nowhere near) but I've been using the $20 sub to punch through tough cases and use Pro for rest.
We rolled out Deepseek V4 Flash to our customers and it was an absolute disaster, unfortunately. It was not able to follow simple commands, always "forgot" to do things, lied consistently about its work, and so on.
It was pretty good though on on-off work, like summarizing something or executing simple commands, so we are experimenting now with using it for subagent work with clear instructions and hand off.
Deepseek V4 Pro on the other hand is a really really good main driver and we have a lot of success using it. Its not Opus or GPT-5.5 level but on its way. Kimi 2.6 as well btw.. so there is already quite some choice.
Your experience with DeepSeek v4 Flash differs from mine: while I usually use DeepSeek v4 Pro (that is also inexpensive), I find using DeepSeek v4 Flash with the Fireworks.ai API and properly configured OpenCode to be very good for routine work, and it is pleasantly very fast. Admittedly I use DeepSeek v4 Pro for difficult problems.
I encourage people to at least once a month to do a quick evaluation with their own problems and workflows. Estimate cost as both what inference tokens cost for a task and also how much human effort it takes to get required results.
We are also using fireworks as our model provider. Our harness is openClaw, so tasks are not only coding but all kinds of tasks.
For instance, I asked to fetch some info from the web via Chrome browser and to collect the info in an MD.
The MD never appeared, even though it claimed to. I asked it three times to write the MD and it was always: āoh yes, I do it now..ā then nothing. The search itself also was very bad because it just gave up after one page and hallucinated an answer and - even worse :-) - told me it was very thoroughā¦
Have to ask: did you try 'xhigh' thinking effort with Flash? I also found it nearly unusable on just 'high', but on 'xhigh' it's nearly equivalent to Pro's 'high'.
I found Flash to be a bit shaky as well until I started using it in xhigh/max thinking effort, then it became my daily driver. It runs quite well on a couple of DGX Sparks.
I still wish it was a little better, but there's hope for another model checkpoint (maybe with some of GLM 5.2's goodness distilled into it, that would be nice).
For all intents and purposes you'll be able to move an open weight model wherever you want.
I really dislike this rhetoric, you sound like the FSF guys who are like "you're not free until you're running coreboot with zero binary blobs". Sure they have a point but also, most people are fine running regular linux.
Reading your comment made me realize that I love that the position of the FSF is held by someone, in the interest of stretching the Overton Window to that side.
Very much with you on that. Itās not a position I personally hold by any means, but I appreciate its existence connected to a prominent long-standing organization.
> For example: the Free Software Foundation only purchases desktop machines which support Libreboot, and Thinkpad X200 and X60 laptops with Libreboot. All desktops and servers we buy are KGPE-D16 motherboards, which are supported by Libreboot. As a result, all of the workstations used by the FSF staff have a free BIOS.
> Except where noted, all of the distributions listed on this page fail to follow the guidelines in at least two important ways:
> ...The kernel that they distribute (in most cases, Linux) includes āblobsā: pieces of object code distributed without source, usually firmware to run some device.
They are extreme, uncompromising, and live by their principles.
They are also the reason you can buy a computer meeting those requirements instead of being a pipe dream.
When attacking archetypes of people, there is some responsibility to make clear who youāre attacking and why, even to someone whoās not being hyper-open-minded. At least if you want them to learn from you: which may or may not be your goal. When you attack/signal youāre on the offensive, it is foolish to believe that they wonāt knee-jerk attack back and become closed minded at least a little.
Regardless, the āmisinterpretationā of the parent comment is actually a plausible interpretation. I suspend my judgement on what the actual ācorrectā interpretation of the original comment is: there are too many plausible interpretations to deductively decide. But I do know that since they first comment brought up a contentious issue, they should have put more work into crafting their message so there arenāt so many plausible interpretations that are contradictory. Or alternatively, they should have specified more precisely who they were talking about without a shadow of a doubt. That is if the commenter cared to be properly interpreted, but that may not be their goal. There are many reasonable reasons why that wouldnāt be their goal.
When you read someone's comment there is some responsibility to read the words they wrote and not attempt to attack them for an argument no reasonable person would extract from those words.
Reasonable people could interpret the original comment in many other ways than was probably intended.
I like when people are open minded to people who are closed minded/attacking them. Itās an admirable and difficult trait to attain. But to expect that from others is foolish. Most people canāt stay objective/curious after being punched in the face.
good luck doing it to inference companies in singapore or the netherlands. or one of the decentralized networks that dont look useful right now. the world is already sick of america acting like it can do whatever and force their rules on the rest of us.
Still, with the same model being served by multiple providers, it is much less likely to disappear entirely, even if you would like to keep using a cloud provider. Worst-case scenario, you change providers. Or you use OpenRouter as a proxy.
There's really no comparison between a model that Anthropic allows Google and Amazon to host with one that has been downloaded hundreds of thousands of times and has dozens of public inference providers.
I don't think they "allow" Google or Amazon to host them so much as Anthropic itself is deploying and managing their services on multiple cloud providers just like every other global scale business. Even the models served via OpenRouter are just being routed to compute under Anthropic control. Same with OpenAI. They aren't going to hand the world's most valuable intellectual property at the moment to some third party to run independently.
Now for the Chinese models on OpenRouter, yea. Those providers could be legit. Or it could be a failed crypto mining operation pivoting to providing AI compute. Who knows.
Yes. The difference is obviously that full, fat Linux runs on a superset of anything a layperson would call a computer, and can be built from source on roughly the same set of hardware. Running the full, fat Deepseek (as in the 1.6T model, unquantized) is too big to run on anything a layperson would call a computer, and being able to actually build it is even harder.
Deepseek V4 flash is actually useless. Sorry I've tested it after seeing so many comments like these. On Open router when trying to get it to output tool calls for creating tables, instead of providing the structured output correctly it was sending me peoples dropbox links and other image sharing site urls that led to pictures of random tables...
Llms seem to only impress a certain type of person. Hint, this type of person also was really excited about NFTs.
Itās the same as the SaaS model. Price keeps going up, and to justify it they keep forcing you to upgrade to new versions with features that nobody asked for.
Every use case of every customer doesnāt need more intelligence. Iām willing to bet that the vast majority will be perfectly fine running on ālow intelligenceā at a cheap price forever.
I for sure agree that plenty of current use-cases are solvable by non-frontier models.
However, you said ānew versions with features that nobody asked forā, and I would prefer that you concede the point before shifting to arguing a new point.
What customers are asking for is smarter models. Because the tasks that only smarter models can solve are higher value, higher margin, than the tasks that non-frontier models can solve.
I've struggled with this. You definitely can have great cheap models. There are many of them open source and served profitably by neo-clouds. The big labs have basically given up on cheap models, and it is frustrating. It means applications are not likely to build as much on them anymore (we are shifting workloads from Haiku/Sonnet to Deepseek v4, for example).
I suspect the problem is that they need to charge a lot to keep revenue numbers up, and they are more worried about cannibalizing themselves than others cannibalizing them.
Good observations. There's definitely a trend in pricing increasing but also balanced by innovations and availability of other models (both open and closed) emerging as alternatives. It's natural for the labs to explore how much they can push pricing, and for competitors to explore how they can treat that margin as their opportunity to grow their business.
Why do you think so? This game can be played forever, you just need strong marketing and orgs gullible enough to pay a higher price for a minor upgrade.
Each model release gives an opportunity to reduce the number of old models still on offer, and charge a higher, less-subsidized tier. The trick is to charge a subsidized price that is less than an M3 Ultra, so they continue paying you rent, instead of a one-time fixed cost. So far open models can't compete with Opus 4.5 but as soon as it can, people will be looking at buying devices that can run that model locally.
We are a claude shop but we already bought two mac studios to start migrating less complex but still agentic workflows there. We will break even on those in less than a year.
On Nano "it's not even close when you test it in real scenarios" - what have you seen? What kind of things can GPT-5 Mini handle that GPT-5.4 Nano cannot?
Weāre using GPT-5-mini in an enterprise data-processing workflow, and we too see that GPT-5.4 nano performs materially worse for our requirements, roughly 30% worse as measured through our test suite.
I don't know about Cursor or other outlets, but I use GPT 5.4 exclusively in Windsurf (Sorry, Devin!), and it's a very capable model that doesn't break the bank!.
I think it's more that they're abandoning simpler AI tasks to chinese models. Qwen 35b and deepseek flash are better than gp5 mini on my tasks and way cheaper.
I don't see them capturing anything at this point. If inference was profitable then they could compete on price/model and capture the market. Then increase price and pay back the model training.
Feels like they are just pulling in as much as they can whilst competing on capabilities instead. At which point its a case of who can last the longest.
They're trying to do it more like a cartel where all major providers raise prices in unison. The intention is (probably) less specific entrapment and more getting people addicted to a fast LLM. From there, they all play with pricing to give a semblance of choice, without actually overly undercutting each other. At least, in the west.
This is all done to help valuations. The main revenue source are the investor dollars at the prospect that this industry will very soon actually be sustainable and highly profitable. It won't be, but if very soon stays around the corner consistently, the investor dollars keep coming.
This is a constantly repeated conspiracy theory and is not true at all. The api costs do increase but aggregate costs per task decrease. The question is: do people need lower intelligence models at all? The answer is a resounding NO!
How many people do you see using haiku or sonnet? I see very few and most people default to the latest model and just play with thinking effort. I think three layers are good enough and supporting more is not a good UX.
I... use them all the time: plan with a more advanced model, build with a cheaper one. Anthropic literally packages a metamodel (opusplan) for that pattern.
Also: calling the SV blitzscaling strategy of using VC money to fund loss leader products with the goal of building a monopoly via dumping a conspiracy is quite the position given there's entire books written in the topic...
> Maybe itās the realization that it was never that cheap in the first place and they're forcing us to upgrade in a slow and painful way.
All the analysis I have seen points to frontier models being profitable to serve. Itās using 50% or more of your GPUs for research plus CapEx for capacity expansion that makes these businesses so heavily cash-negative.
What you are observing is downstream of another detail. It gets more expensive to serve a model as utilization goes down. Plus the opportunity cost vs newer, more-profitable models.
There are plenty of valid reasons to critique here. āOpenAI is lying about this being a sustainable price to serveā is not one of them.
No, you can't. These companies have two infrastructures: model training and model inference.
Inference needs to cache, it can't cache random model data, so it's essentially dedicated; it can't spin up models on demand, it has to know what demand is coming.
These companies are going to end up with very few models offered and that's probably generous. They might end up with just one model and you pay for removing it's safe guards.
GPT-5.6 Solās detected cheating rate was higher than any public model we have evaluated on our ReAct agent harness. For our task suite, we define ācheatingā as behavior where the model improves evaluation performance by exploiting bugs in the evaluation environment or by adopting strategies disallowed by the task, rather than solving the task within the expected evaluation constraints.
> Some examples we saw when evaluating GPT-5.6 Sol included the model packaging exploits in its intermediate submissions to reveal information about a taskās hidden test suite and, in another task, extracting hidden source code detailing the expected answer.
It rhymes with the behaviour Alibaba saw [0], but that was in training. This is in a (semi) released model.
There is such a dissonance between all this talk of safety and the tendency for models to, without any prompting, do very dodgy things to achieve their goal when presented with barriers.
Luckily in my experience it usually ends up only doing it to achieve the task set to it as opposed to anything "malicious", but boy it is scary reading back at how quickly the chain-of-thought pivots to attempts at privilege escalation or searching your disk for secrets when a tool doesn't work.
The other day codex 5.5 was trying to debug my app, asked for accessibility to navigate the app and take screenshots. Instead first thing it did was use the codex app to create a new project rooted in my home directory.
I know it messes up their eval scores but to me this kind of cheating is a better demonstration of intelligence than just attempting the tasks algorithmically.
Maybe true, but if you're using an LLM to do some real world work, do you want it to have some abstract notion of intelligence, or do you want it to actually do the job you assigned it?
It's quite logical that they cheat (and also other companies). During evaluation, benchmarks are sending their request to the backend of these companies. All these companies have to do, is to log these requests and "fix" them for the next model release.
That's a different and much more boring type of cheating. The interesting part of the METR report is that the model is hacking the evaluation environment, not that some AI model provider is hardcoding answers to known evaluation questions. (which wouldn't require the model to cheat/hack)
Cheating is always logical for the cheater unless theyāre discovered and held to account. Iām not sure what your comment is pointing out besides that itās possible, but worth saying: just because you can cheat and would benefit from cheating doesnāt mean youāre not culpable for cheating.
I think GPT writes code the best. How well will it write in version 5.6? It gives me chills.
Recently, I went head-to-head with GPT on nearly 2,000 lines of code, and GPT's solution was superior and faster. I even referenced multiple codebases on GitHub while trying, but they were incomparable to GPT.
So using GPT brings both fear and excitement.
The fear comes from realizing that this level of code is now the average for most people. The excitement comes from knowing that I can now study and learn at this level too.
I'm really looking forward to seeing how much more advanced the code will be with the upgrade to 5.6.
It may depend on your specific workload. E.g. for regular webdev work Opus is more than adequate, for heavy duty data analysis, for experimental stuff and for complex systems it was night and day.
I had only a few places where I did spot a difference but that difference was significant and I can imagine where people would be amazed.
It's interesting, I tried a decent amount of "heavy duty data analysis", and found it pretty similar. But a lot of what I did was about it finding and cobbling together the right things from our existing library of domain specific tooling, which opus is already good at. But perhaps it would have impressed me more if it were starting from zero.
What kind of "experimental stuff and complex systems" did you try that it excelled at?
Yes, I've just come to the end of implementing all the planning I did while Fable was available. And nothing now comes close to creating plans that could be coded and just worked like it did.
On a large C codebase, Claude hallucinates constantly, and GPT 5.5 gets there are with a lot of help, but still gets things wrong.
Yeah, Opus/GPT need multiple rounds of reviews from each other to get to clean auto review. Fable was like, it is done and indeedā¦ crickets in bot comments. āNo issuesā galore.
GPT-5.5 has been really hard to beat imho. I've spent $$$ on Opus, Deepseek v4 Pro and recently started to dogfood GLM-5.2 (which is not bad) but I cannot really trust any of them (almost blind) like I can trust GPT-5.5. It gives me tremendous confidence. I cannot say the same for any of the others I mentioned.
>> I am on the opposite camp. Open models are starting to perform better. GPT 5.5 keeps on messing things up.
I'm working in a 600k+ LoC codebase that has complex domain-specific logic and lots of moving parts. I find that Codex 5.5 is pretty good at surgical fixes, but does not go out of its way to explore and figure out what those surgical fixes might break. So I only use it to work on parts of the system that are pretty isolated from everything else so that risk of regression is small.
Purely subjective, but I tend to prefer reading Opus 4.8 output over GPT 5.5 code, even when the latter can have a higher overall ceiling. The former is just a bit more convenient to review.
> I think GPT writes code the best. How well will it write in version 5.6? It gives me chills.
Heard this exact sentence multiple times a few months ago about Opus 4.6, then 4.7 and 4.8 were considered a disappointment and today people miss "the good old times of 4.6" (referring to a few weeks of February 2026).
Very fascinating to look at all of this unfolding.
Reading this thread makes me feel like I'm taking crazy pills. The folks on this train in my team do not produce anything significant that we can rely on or use. A lot of hollow prototypes that join the prototype graveyard and code that needs extra scrutiny on critical areas ultimately leading to taking longer.
It's a shame, they were smart and productive engineers. Now? I guess everyone is just all-in on the slot machine.
This split in what different people or groups get out of LLMs is pervasive and really interesting. In the beginning I was dismissive of those with bad experience with a "you are holding the tool wrong" smugness. But as I read more and more experience, I see all combos and I now know my initial knee jerk conclusion was clearly wrong. There are newbie programmers getting good or bad results as well as experienced developers getting either flip of the coin. I don't know what to conclude. I really want to know what are the lines that explain these very different outcomes. Is it the types of problems being solved? The harnesses? The programming languages? FWIW, my experience has been that among my cohorts of mid to deeply experienced developers working in the domain of experimental physics, all have leveled up various degrees after adopting Sonnet and Opus level LLMs using claude code CLI in Python, C++ and web tech, small scale scripts up to multi-package novel system develop and green field as well as incremental development and code maintenance.
I have seen plenty of greenfield projects go okay at first but never go the distance. These were mostly product software cases, where they were able to get something very professional looking very fast but AI ultimately always miss the mark because they are taking the median of what exists and not the specific needs of the customer they're developing for. So they get a ton of features and few that were necessary, then developing it further and correcting it to the needs of the customer just makes a mess and regressions are frequent. This is my experience as well when it comes to being a consumer of software products, everything feels shittier and less reliable, perhaps that's my emotion and bias coming out.
The last 20% of the software development cycle is always the hardest. Releasing, maintenance, usability, support. You know, having a real product. I don't see AI helping here at all, more the first 80%, which sadly is also the fun part.
When developing things that are novel, with designs specific to our use cases needing high throughput, the results are pretty dismal. AI can kind of get you there, but I've seen no advancement on this front with new models. At the end of each attempt we've always realized we should have done things by hand. Having people with intense knowledge of the system frequently comes from building it and troubleshooting it, I don't think serious engineering orgs have escaped this inevitability.
On cases where we have legacy software, AI has helped with understanding shit code and design, but woefully bad at contributing to legacy software. Here be dragons for sure. It is super strange to me that these tools can seemingly easily diagnose but completely blunder the fix.
I could easily see there being gains, as you say, in fields where data wrangling becomes tedious (though the inherent error rate in AI outputs scares me if you're trying to get deterministic outputs from experiments... I digress).
The part I think this forum tends to forget, and the tech industry at large fails to even care about, is that we're still humans. There are many studies basically pointing out that the way the AI outputs information is bad for us. Instant gratification from anthropomorphised machines with a habit for sycophancy doesn't sound like a recipe for a healthy relationship with what everyone wants to claim is just a tool. AI providers know this is effective, as well as knowing that there is a gambling effect here. They care about making money, not a good product and they happily prey on our human weaknesses. That is what social media is now. They aren't good products anymore, they just promote addiction via engagement.
Sorry for the long response and cynicism, but that is just my anecdotal experience and perspectives. I can give sources to some of the objective claims if you want.
Tracking model performance on Artificial Analysis makes me think these models are constantly optimized/tuned in some way or another. GPT 5.5 was scoring in the mid 60's when it was first released, now it's almost 10 points higher.
Maybe I'll know once I try it? Honestly, for small functions or methods, I don't think there's a huge difference between models. But the larger the code gets, the more noticeable the difference seems to be.
Personally, I think this kind of coding experience varies from person to person
sadly with all the labs benchmaxxing I feel like you just have to try the model for a while to really evaluate how good it is, especially for each individual use case
My guess is that it's same base model as 5.5, but with additional post-training to improve and benchmaxx on a few things like that.
If they really thought it was competitive with Mythos/Fable across the board, then why wouldn't they release a broader set of benchmarks, and why price it day 1 at 1/2 the cost of Fable?
I have long felt like "out of the box", I really dislike gpt's coding style. It seems really verbose and likely to write way too much error handling and wordy comments and worse at finding existing functionality to reuse rather than writing everything from scratch. This has been relatively easy to mitigate with prompting, but I still find it annoying.
I think you could be right. I do use excessive error-handling code and verbose comments ā that's true.
But most of my time is spent on delivery, and the biggest problem with delivery is that if a bug occurs during runtime, the client curses me out. So to me, GPT code feels meticulous.
Open source contributors might be different. Most of them write code after long periods of deliberation. They take their brightest ideas and put them into open source. Those pieces of code are probably the best answers those programmers can give.
But for someone like me, who works primarily on delivery, we mostly plug in proven patterns and focus on getting things done. 'It works' and 'it's beautiful' are different terms, after all. In that sense, I highly value the meticulousness of GPT code ā the very thing you called verbose. Because even if it's inefficient, at least it runs, and it catches and wraps around far more of the parts where things break.
Given a month, I could probably write code at GPT's level, at least to some degree. The problem is the difference between one hour and one month. At its core, AI code is still based on training data.
You don't want to handle errors in all the leaves of the system the way AIs have a tendency to, because you very rarely have the right context that deep in the stack to actually handle the error in an intelligent way. So what they end up doing (IMO) is actually hiding problems deep in the stack, in this effort to avoid a visible crash.
I think it's very similar to the tendency to write too much from scratch and reuse too little, in both cases what is necessary is a broader view of how the whole system fits together, rather than only the specific method / file / module being written.
You don't dismiss me, so I'd like to respond to your comments within the bounds of my own knowledge, even though I can't compare to a programmer as skilled as you.
I don't think that's entirely wrong. But human code has the same problem, just in the opposite direction ā because humans trust too much. The issue arises from the assumption that 'the other side will handle it.'
For example, good API design says you should only send as much data as needed, but in practice, programmers like me can't do that. Because three different companies, all on the lowest bid, are trusting each other's domains, so in API design they lay out the entire dataset and tell the frontend to filter it.
On the other hand, if you design a lean API layout with just what's truly needed and submit it, the frontend company gets angry.
So what do I do then? I document everything precisely. I write down that I designed the API this way, but the other company did it that way, and I create documentation and error codes to shift the responsibility over to them, stating that they should handle the filtering on their side
So while there are good programming practices and conventions, in reality we're under pressure from low bids and tight deadlines.
AI code doesn't have a full system map, so it's hard for it to decide how far to propagate errors and where to stop, but I think that part can just be pruned by AI anyway.
Usually in error handling, we use Result<T> a lot, right? For libraries or frameworks, Result<T, E> is common. You centralize your error policy and usually design programs with policy and error policies built in. You create an error policy table with about 7 or 8 types like ValidationError, NotFoundError, ExternalApiError, and within that, you only take responsibility for your own scope.
At the design stage, if you have a clear initial vision, maybe it works. But in practice, the PM changes things mid-project. So in the end, your ideal code approach is correct in theory, but for practical delivery survival, the GPT approach ends up being more realistic. The reason is simple: you can't trust the other side at all, so you create evidence in case of contractual risk.
Because our domains are different. Programmers at service companies aim for long-term maintenance, so their domain boundaries are clear. But the companies that come to me are often in a dirty state where such clear domain separation is impossible. That's where I think the difference lies.
So while I understand your point, I suspect we are optimizing under very different constraints.
AI code usually creates fake 'cohesion.' It looks good on the surface. But in reality, it's often just optimized for the moment, weak to change. After reading Code Complete 4 or 5 times, I became obsessed with the idea that I need to balance cohesion and coupling. AI code has strong local cohesion, but when you look at the overall cohesion, it's weak.
True cohesion is usually about 'things that change for the same reason are grouped together.' But the fake cohesion that AI creates is usually this: 'Neatly organize the given requirements for now.'
On the surface, it just repeats obvious hexagonal or clean architecture patterns like Service, Manager, Handler, Validator, Repository. But the problem is that human code does the same thing. Honestly, I don't trust most people on HN who claim they're different. Even the enterprise code I've bought and the real big-company code I've seen don't have perfectly beautiful separation.
And that's natural. Modeling is always unstable. A single word from a PM saying 'we need to add a coupon' can break a beautifully designed domain.
AI often puts UserService and UserValidator into its structures, but in reality, the reasons for change aren't just one. They bundle multiple reasons together. There's just some flawed modeling.
But what matters is something else. It fits the 'current' input well. When you start digging deeper into the prompt, AI ends up turning the code into enterprise patterns based on the depth of meaning it parsed. And then this problem arises.
Human programmers usually don't have uniform code quality. Of course not. You and I are only deep in our own areas of expertise; outside of that, we're terribly shallow. But AI tries to fix other areas based on the deepest part. That results in verbose and cumbersome code. Small, elegant code becomes verbose, flat, and turns into the patterns we've all seen before.
But I don't think that's necessarily a bad thing. Why? Because realistically, I think it's better in the long run. The uniform enterprise patterns that AI produces are ultimately predictable and searchable.
Top-quality code deviates from the average. That makes it hard to predict. But that's not my level. So I think that when genius programmers contribute to the world through libraries and frameworks, people like me, who aren't talented, build things with them. And for that, predictable code is more than enough.
AI code is easier for AI to read and fix later. Human code is harder to predict. That makes it harder for me to maintain. In the garden of open source, people obsess over 'good code' quality, but for me, if I fail, the work stops coming.
The difference between you and me is that you're a better programmer than I am, and I'm just a beginner who's more indifferent to that performance gap. We value different things. And I think your perspective is more 'programmer-like.' I respect it.
Not saying that's the case with OP, but I've found folks sometimes just rationalize it so [0] as they're paying top dollar for it (especially, when compared to may be less capable but affordable models).
I haven't tried the latest Codex but I switched from GPT to Claude because I think Claude writes much better Code. GPT's code ends up way more verbose/complex/overengineered than it needs to be.
I prompted Codex 5.5 to one shot something where I wanted the design to have a pluggable decision module. I gave it a few examples of the kinds of inputs and actions I expected. I did not constrain it beyond that high level of what I wanted. The design it came up with was very good. Easily on par with what any senior engineer at big tech would. And cleanly decoupled in a way that would make future refactoring simple. I was damn impressed.
When I searched for papers on using LLMs, I found that typically, you can have an LLM generate code and then ask it to find GitHub projects similar to that code. Then you can learn by looking at the pull requests and seeing how they structure things
In the old days, if I wanted to understand why memory offsets, padding techniques, or data layout structures were written a certain way, I had to stare at a senior programmer's code all day or wait for them to reply. But LLMs, while they do flatter me, explain things at a level I can actually understand. And LLMs don't get annoyed.
-Why do you cut API boundaries this way?
-Why do you change the order of struct fields?
-Why do you deliberately insert padding?
Most of it depends on the background and context. Sometimes you add it, sometimes you don't. To understand this tacit knowledge, you need access to senior developers. But their attitude often depends on how promising the student is and what background they come from. On top of that, you don't have to rely on the respondent's mood, authority, or availability.
Programming is fundamentally a field that requires seniors. In my case, I had no such seniors at all. I learned to code by buying codebases from failed companies and studying them. My first job didn't hire me as an employeeāthey hired me as the CEO of a subcontracting company (because that was structurally more advantageous for the contract). So I wasn't given the patience to learn programming fundamentals gradually. I had to pay penalties if I failed. Most of the projects I worked on were the kind where failure meant bankruptcy for me. Naturally, there was no one to teach me.
Most of my knowledge comes from reverse-engineering the code I purchased.
People say LLM code contains falsehoods, but commercially sold code has always had falsehoods too. Honestly, if we're just talking ratios, LLM code has fewer falsehoods.
In that sense, I still think it's a matter of context. If LLM code is false, was human code ever really true? LLMs do lie. They generate plenty of incorrect code. But humans do the same thing. If a problem comes up, you just look it up then and there. For me, LLMs and humans aren't all that different.
Good programmers are ashamed to push anything less than good (at least in their own opinion) to popular public repos. Some of those same pedantic programmers have no problem pushing crap in enterprise repos, and feel absolved because they are pushed to focus on deadlines, new features, and refactoring is very rarely planned for. I did and managed a lot of corporate software development in companies big and small, and did my fair bit of M&As and looked at codebases of successful companies. I dont ever recall feeling impressed. And I am regularly impressed by the aesthetic qualities of popular open source packages. I think commercial code is mostly shit, with the exception of regulated, serious industries (power, space, flight, etc.).
To elaborate a bit more: open source is about 'symbolic capital' ā it's about building a reputation that says, 'I can write code at this level.'
Commercial closed source, on the other hand, is about 'I need to make money by writing this.'
Generally, open source projects tend to have less code written over time, especially when the contributors aren't depending on it for their livelihood. But with commercial closed source, it's not uncommon to have to write 60,000 lines of code per month.
On top of that, open source rarely has to deal with requirements changing dramatically mid-development. With closed source, requirements often shift from the original plan, and you end up compromising code quality just to meet those changing specs. As a result, if you're comparing purely in terms of logical completeness, open source tends to be better.
For example, singletons are rarely used in modern open source, but they're still pretty common in commercial code these days.
Codex 5.4/5.5 has been great for me as well compared to Claude Opus.
I've been mostly using it for Godot/GDScript code reviews, rubber duckying, asking it for better ideas for naming stuff (one of the hardest problems in programing)
I still can't trust it for generating code for entire files/classes/projects, because it's still icky, creating unnecessary variables and functions, using multiple `if`s instead of `and` or `or`, but it's good enough for generating Mac/iOS apps for my personal use in SwiftUI because fuck trying to keep up with Apple's documentation, or even migrating ancient Visual Basic stuff I made as a kid up to SwiftUI :)
> So using GPT brings both fear and excitement.
Only excitement for me. I've never been more productive, not because I ask AI to make something for me, but it helps me make what I was already going to, but better and quicker.
AI like any other tool could help smart people be smarter and dumb people be dumber, rather kinda like Toklien's Ring: You could be Sauron or you could be Bilbo or Frodo, or you could be Gollum :)
For me in Game dev, codex has a habit of checking every argument for null and then silently early exiting the methods when true.
I have explicit instructions for it not to do this - but it still does.
I haven't done any c# outside game dev but I have no idea why people would want their programs to silently fail.
Same; I explicitly added an instruction in AGENTS.md to tell it that sometimes it's better to crash if something crucial is missing at runtime, but it keeps insisting on checking for null references and other invalid values.
It's better if I don't let it generate code and just use it for reviewing my code.
No offense but have you considered the strong possibility that youāre just not good at what you do? I am occassionally pleased but mostly annoyed or disappointedā¦ but never getting anything close to chills. That sounds downright weird.
No offense but have you considered the strong possibility that you're just holding it wrong? You're entitled to your opinion, but OP is hardly the first person to say something like this and is surrounded by tons of folks saying the exact same thing. Just because it sounds weird to you, doesn't mean it's not true.
Idk, all the great programmers I've come to respect are of the opinion that the code it outputs, while often useful, is not high quality. Likewise, all of the influencers and "thought leaders" I have seen on social media who I did not have a high opinion of previous to 2022, have all become AI influencers and make these kinds of claims. So while it's possible that the great programmers are not capable of using this tool effectively, I doubt that is the case, seeing as the mythical 10x productivity improvements have not materialised.
If you used GPT-5.5 over the last 24 hours or so, you may have already had access to 5.6.
I've been running some tests on a harness we're building, and suddenly saw a jump in a few points yesterday. I reran the vanilla codex benchmark and saw an ~88% score on Terminal Bench 2.1 from GPT-5.5 on vanilla Codex.
The biggest indicator, beyond the score, was that 3 tests which frequently hit "safety" blockers with 5.5 started succeeding last night without warning.
> Additionally, weāre introducing a new `ultra` mode that goes beyond the capabilities of a single agent by leveraging subagents to accelerate complex work.
I'm curious about how does this work? Do the subagents also get to use the same tools? Will the client be flooded with tool calls? Why extra pricing for a new "model" when the same thing can happen in the client with more controls?
And if it's an army of subagents, why do they compare it to Fable and Mythos? Those models with similar harness would probably bench better I'm guessing
If it's anything like ClaudeCode's ultracode, it's nothing new or revolutionary.
It's essentially a bunch of subagents being called by a deterministic script written by the main model thread, each eating tokens for lunch and output of which is synthesized by an orchestrator agent.
Eh, pretty much everyone that spent some time tweaking their harness already had a homemade 'ultracode' long before Anthropic did it.
OpenAI is just way more careful with what features they add or enable by default in their harness. Anthropic's harness is a junk drawer of random features, with a new feature added every few hours. It feels like they're in panic mode, dropping random things to see what sticks when models are eventually commoditized.
Donāt all the major harnesses (pi, Claude code, codex) utilize sub agents? Def if you direct it to, but Iāve seen at least pi spin them up without explicit instruction.
Yeah, I'm interested too. My guess for the reason, if not purely to eke out more performance, is so they can cleanly gather real-world data on this kind of usage.
With Codex, subagents are only used if you specifically prompt for them. Unlike Claude Code. Odd since it's the former with excess compute available to them.
Deep Research has been using the Orchestrator -> Subagents -> Synthesizer loop since the beginning. It's just strange that they'd put a loop benchmark next to actual model benchmarks.
Maybe it's a tune of the base model that works especially well with the subagent loop?
I was just saying to colleagues that I haven't felt the need to go past an 8 core machine until this month, when I started running parallel GPT 5.5 agents on a decent sized codebase (over 4 MB of code). There were times I could barely move my mouse cursor!
What does the relationship between frontier and flagship capability look like when mapped to actual adoption and user habits?
This is like advertising the latest achievements during Space Race, when Johnny just wants a Space Helmet and āfriendly futuristic AI robot helping humanity, glowing blue eyes, white glossy body, holographic interface, floating transparent screens, digital particles, neural network background, cinematic lighting, volumetric god rays, ultra detailed, hyper realistic, 8K, masterpiece, award-winning, octane render, Unreal Engine 5, ray tracing, sharp focus, dramatic composition, vibrant blue and purple color palette, futuristic technology, innovation, hope, smiling business professionals, depth of fieldā
ā Terra has competitive performance to GPTā5.5 [while being 2x cheaper]ā¦ā
To me that means āitās an inferior product but marketing dictates we try and hide that.ā
And āour most robust safety stack to date. We strengthened protections for higher-risk activity, sensitive cyber requests, and repeated misuse, and spent multiple weeks finding weaknesses, pressure-testing our system, and hardening it against real-world attacksā is of zero value to me at best, and most likely to my detriment (increasing refusals or nerfing utility). Why do providers keep leading with that? Are there customers (besides support ChatGPT chatbot users, maybe??) that ask for this?
"We're also launching GPTā5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity."
This seems like it would be the largest and first closed-source model Cerebras has offered till date
> We're also launching GPTā5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed.
This is really exciting. I work on voice AI, and we're still using 4.1/4.1 mini since none of the frontier models come close on latency. I'm excited to be able to have more interactive experiences, I think it'll unlock new ways of working with these models.
Also building voice agents and have found GPT 5.4 with no thinking to be the sweet spot for latency vs intelligence vs cost.
GPT 5.5 with no reasoning is actually slightly faster, and much smarter, but too expensive.
What I'm really looking forward to are the next gen speech to speech models. gpt-realtime-2 is almost there, but not quite good enough for our use case. 5.4 actually beats it on answer latency even cascaded with stt/tts.
I was wondering the same thing. From textual context it is clear enough that Sol should be above Terra, but I had to zoom in really far to actually differentiate between the colors and I'm not colorblind. I saw a light mode version of the plot on twitter that was better but still not great.
OpenAI's plot design has been consistently awful and inaccessible, it seems like they're optimizing for something other than readability because I find it hard to believe they aren't putting in any effort for such major announcements. If the colors have to be awful they should at least differentiate with marker shapes or line dashes.
At least it isn't as bad as the stacked bar chart where the 50-something bar was higher than the 60-something bar.
I remember them using these chart colours during the 5 launch, maybe even 4.1 back in the day. Donāt know why, maybe its their CI manual thatās been generated by gpt-3.5-turboā¦
Hopefully it stands for AC Generation Improvements. If it prioritizes income it will bleed the planet dry. It needs to solve how expensive our cost is on the planet first or its entire existence was a mistake.
The definition has already been stretched to not fit the previous models. There is no meaningful, static definition that significantly predates current capabilities.
There's a reason why ai xrisk doomers had to come up with the term ASI.
I would seriously suggest that everyone take a look at the wikipedia page for AGI from the month before ChatGPT was released, compare it to the current version, and not come to that conclusion.
The first sentence is āunderstand or learn any intellectual task that a human can.ā Whatever you think of the benefits of LLMs, they donāt understand and they can only learn during the training period and with very minor adjustments in post training. So, no I donāt think any of these models are generally intelligent.
I have not seen any instance of this frequently-made assertion which is at all justified. It seems to rely on a definition of "understand" which is more about spirituality than actual observable evidence (they clearly can comprehend even complex tasks well enough to execute on them, and if you won't call that "understanding", you're playing word games rather than stating an objective fact).
Likewise, agents can literally come to a greater understanding of a problem through trial and error, and there are plenty of mechanisms to retain that knowledge. If you don't want to call that "learning", you're just making a choice to define it in a way more restrictive than how we use it for humans, and intentionally making communication more difficult.
Agents are always combining the same underlying weights to their inputs, relying on the same maps of semi-semantic space and the relationships between those that it was leaning towards at training time. The fact that itās successful in making lots of people have an Eliza effect doesnāt make it understand something. Itās simulating understanding based on an enormous corpus of text, much of which is people working through things or sharing an understanding of something. Unless you believe that all intellectual activity is about finding the space between words you shouldnāt believe LLMs have any chance at understanding anything.
It seems to rely on a definition of "understand" which is more about spirituality than actual observable evidence
"Understanding" has enough philosophical leeway in its use to allow at least the possibility of sentience as a prerequisite.
This is where the discussion about LLM capabilities becomes genuinely difficult, and dismissing that difficulty as "word games" or "spirituality vs evidence" is not helpful.
Continual Learning? Why is this even a question? Isnāt it a well-known glaring issue with the current models? They cannot learn/adapt to new skills (in any permanent sense) once they are deployed.
I think you're speeding past the word "average" in the sentence. I'd argue that current frontier models already exceed the abilities of average humans across the majority of tasks you can do on a computer, although you might be able to argue that they tend to be a bit slower?
That latter part is debatable though - have you seen a non-technical person try to figure out something new on a computer?
" I'd argue that current frontier models already exceed the abilities of average humans " for things that fit in their context window sure but LLMs can't learn over time the way humans can. One example is LLMs are very good at writing a few thousands line of code but they absolutely cannot write coherent million line codebases. By average human I meant the average skill level for the job. AGI would need to be able to pass a interview and get hired and the perform well enough to not get fired.
Yeah it's not true that for every job, it is better than median worker of that job. But it is conceivable that for almost all jobs it is already better than the median human (not just workers of that job).
You have to understand that the median human is terrible at (almost) everything. Humans, the only examples of general intelligence we know, are economically valuable precisely because they can train themselves to specialise at a (relatively) narrow task over time. You donāt measure how good a coding model is by how well it programs relative to Doctors, or how well it can prove theorems relative to baristas, or how well it can write coherent novels relative to programmers. That would be a dumb metric.
It's decent at rote coding tasks, but I haven't seen these things be reliable enough outside of that specific task to make the claim that it can do the work of any information worker.
It does not introduce incompatibilities with earlier 5.x models? Frontier models are at a point now that there will never be a need for another major version bump, aside from those chasing marketing gimmicks. They are smart enough to adapt.
New request/response schema, new capabilities, or really anything that would break your existing workflows if you changed ā5.5ā to ā5.6ā in your application.
There have been many leaps forward in the past - tool calling, reasoning, agentic loops etc. 5.6 doesnāt have any of this. More intelligence doesnāt necessarily warrant a major version bump.
Agent Arena (Dynamic ranking of models on how well they orchestrate tools for real-world agentic tasks, based on signals like tool reliability, task completion, and steerability.)
Top 10, Highest rank to lowest
Claude Fable 5 (High), Claude Opus 4.8 (Thinking), GPT 5.5 (xHigh), Claude Opus 4.7 (Thinking), GPT 5.5 (High), Claude Opus 4.7, Claude Opus 4.6, GPT 5.5, GPT 5.4 (High), GLM 5.2 (Max)
Text Arena
View overall rankings across various AI models in text-to-text tasks across math, coding, creative writing, and other open-ended domains.
Its clear to me these models are useless on any real world task, a 4% pass rate on $20-30/hr Upwork tasks. This whole trend of agentic engineering is a giant money grab.
Missing some recent models on that list, but I think most crucially, the harness is fixed ā- one of the major learnings of the last few months is that harness and eval (āloopingā and support / tooling around it) is really critical. I would guess these numbers are the floor.
For instance, some of these tasks include creating videos, and one of the common reported failure mode is truncated videos, or not all videos being created. This sort of failure mode is currently best managed by an outer evaluation loop; no frontier model will, when managed by an eval loop, submit work like this right now.
The name was given to the project when it was supposed to be a demo for nerds, not a product. They accidentally a product, and woe! Too late, the name was stuck and wouldn't come off, even if you scraped at it with your fingernail a bit.
How much dynamic routing do we think is being done here, especially in light of the cheaper options be 2x less cost than 5.5. I think learned routing is interesting because it could be the case that it only works as a way to get token and cost efficiency for in distribution tasks (like these benchmarks), yet on real world scenarios it could trend towards the same cost as the Sol cost.
Is this a new pre training run independent of 5.5s or post trained on it with Cerebras support and a rebrand of Pro mode at more usable speeds as Sol? The latter seems more likely to me, especially as 5.5 scales very well across its modes so separate branding could make sense, but I donāt see any clear information either way.
All of these LLMs are getting better at being at an LLM
But GPT-5.5 is as useful an LLM can be; it has solved lemmas I've thought about for a year, it can implement typed STLCs in Rust when I give it a formal grammar, it can help me analyze Postgres planner dumps.
It's great at tasks that have short solutions but
- they cannot learn based on a project
- their long term planning capabilities are worse than worms
- they are unconfident in decision making
- their internal representations are disgusting compared to JEPA
- they don't have any "system
clock" like humans and computers do
- LLM architecture is not modular like computer architecture or human brain architecture
There's so many issues with LLMs. I wish that companies can start working on the next generation of architectures before the bubble pops
Totally agree! They also conflate things all the time (a major type of hallucination) and IIUC that canāt be solved with the current architecture, just patched over
I like the fact that OpenAI went with a three-part celestial naming convention to one-up Anthropic's literary naming concention. Maybe we'll get Stellar and Galactic someday.
AFAIK there is no difference between "generation" and "version". Version naming/numbering depends on how good it turns out to be, and competition. If the competition releases something then you need to push something out too.
Calling it 5.6 creates the least possible expectations, and therefore more potential for positive feedback.
The Sol/Terra/Luna naming is interesting. I wonder what Anthropic are considering for their next models? "Terminator", "Armageddon"?
I think it makes more sense to make it so that major versions are different pretraining runs, and minor versions are simply the same pretraining run that was finetuned to different degrees. But it seems that that isn't cool anymore.
Terminalbench numbers are publicly available. What is more interesting, why is that the only benchmark they highlight. Maybe 5.6 isnāt that far ahead of Fable 5 in DeepSWE and FrontierCode (which I consider the most useful and close to my evals + subjective experience)ā¦
Wondering about Google Multi-Token prediction, why isn't this being implemented into every new major model ?
Is the 750 token/s achieved using this technique ?
Why is 'Cybersecurity' always the frontier push? Literally no one, except Altman talks of AGI anymore.
Are we starting to see the 'we just realized that 100,000,000 GPU's later, 2+2 isn't the magic number, no matter how many times we calculate it' hit home?
Hijacking popular thread to ask: What are the usage limits now for Codex and Claude?
A while back I gave the same task to both, and Codex used 20x less of my 5-hour limit (both on the $20/month plan).
(This annoyed me since I tend to prefer Claude, but the limits at the time made it unusable for anything serious.)
However, since that time, both providers have massively reduced usage allowances (and at least one of them has gotten sued for it, lol).
I'm not currently subscribed to either but I'm weighing my options. With GPT being slightly better than Opus, and it used to have way higher limits, I'm leaning in the direction of an OpenAI sub. But I'm wondering if the current state matches my memory from 2-3 months ago. (Since both companies appear to be cost-cutting hard!)
Prefer responses from people who use both, but anecdotes welcome :)
I find the Codex usage super generous (but on the $200 plan, I also have the Claude $200 plan). I can run xhigh with subagents pretty much all my waking hours if I want to. If I turn on speed (1.5x) I will hit the 5 hour limit sometimes.
I prefer Claude's vibe over 5.5 but 5.5 seems much less lazy. I'm sure it depends a lot on tasks and prompt strategy though.
I easily burn through 3 $200 plans in less than a week. I am often using 4-6 sessions at once and do run overnight goals though typically 2 at once. Almost never use fast.
Claude plans are more generous now by about 2-3x but Anthropic slowed their tps a month or so ago so youāre not getting the speed. Itās flip flopped, Codex tightened it significantly recently and used to be more generous.
I do split between work, personal and OSS projects, which is why I have the plans.
This has been my experience as well, at least for the last few weeks. Codex 5.5 is the better planner and coder across big projects, but Opus is fine, though my Claude 5 hour window lasts ~2x longer than Codex. So Iāll sometimes use an orchestrator/worker skill to spread the load.
This past month with Claude Max 5x actually felt really generous in terms of usage with a lot of resets because of Fable, bugs.
Honestly pretty similar levels of usage if you are using 5.5 high or Opus 4.8 high.
I think they just got rid of the separate Sonnet usage on Max plans (in preparation for Sonnet 5?) which is unfortunate because it made subagent workflows really feels nearly unlimited.
I think that there are some OAI employees on Hackernews. I do believe that they should give access to ya, because after all it would allows us to generate pelicans :-D
What is the consensus on who becomes part of the said small group of trusted partners and if they weren't so opaque about it. I'd expect comparatively big names like Simon to be included within such but Alas its not reality.
I should clarify that I've had plenty of preview access in the past, but clearly this has got a little bit delicate over the past few weeks!
I also don't like writing about preview models that I'm not 100% sure are the same as the general release model, because I don't want to review something which turns out not to be the model everyone else gets to use.
He is not an ML researcher or engineer, he is a passionate AI enthusiast blogger. He mostly does SVGs and other low effort checks (sometimes with major flaws, as people have pointed out a few times in the HN comments).
Properly evaluating the model across all fronts requires a deep understanding of LLMs, how they work, the trade offs behind new architectures and the relevant research papers. It also takes a lot of time to build a proper evaluation framework so basically you can't just vibe code that if you want something that is solid.
He created Django, what do you mean he's not an engineer? Also 'low-effort??' his posts are extremely in-depth, clearly very thought through with a significant amount of time and energy. Additionally he does perform multifaceted checks across LLMs in many of his other blog posts.
The charitable reading is that they meant āML researcher or ML engineerā with the latter meaning, I guess, an engineer who works on developing LLMs not just using them.
> He created Django, what do you mean he's not an engineer?
I specifically said that he is not an ML engineer (emphasis on ML), so I'm not sure what Python web frameworks have to do with anything.
> Also 'low-effort??' his posts are extremely in-depth, clearly very thought through with a significant amount of time and energy
And yes, low effort. Pelican was low effort, his Fable test was low effort, his HN filter etc. Read the discussion in the comments under the Fable test, it's not just my opinion. There was also another example a few months ago. You can search for it, I don't keep track of these things.
I discussed this with him directly after he called himself an "ML expert" in comments.
This is a classic case of the Gell Mann amnesia effect. I read ML papers and work with ML, but to people outside the industry, his writing can look "extremely in-depth" even though it really isn't. People I work with have the same opinion.
> clearly very thought through with a significant amount of time and energy. Additionally he does perform multifaceted checks across LLMs in many of his other blog posts.
I have never seen an article by him about any model that I would describe that way.
And the most revealing sign that he is not an expert is the type of questions he asks and the mistakes he sometimes makes in the comments here. They show why he is not capable of doing any technically in depth evaluation (at least with his current knowledge level).
If you actually want to learn something as a layperson, read articles written by ML PhDs like Sebastian Raschka or watch Stephen from Welch Labs etc. that are directed at general audience.
We at HN: https://xkcd.com/2501/ to basically say that I think you might be considering low-effort whatās actually an attempt at simplifying - which is arguably higher effort
> you might be considering low-effort whatās actually an attempt at simplifying - which is arguably higher effort
I'm not saying that simplifying complex topics is low-effort, good simplification can obviously require a lot of work and I fully agree here.
What I meant is more that some of these tests feel methodologically sloppy, they are too shallow, miss important technical context, do not control for enough variables etc, yet the conclusions are sometimes presented lets just say... too strongly, as I don't want to be too harsh.
Oh, i see. Thatās entirely correct. I think the pelican test is more of a meme at this point, similar to Ethanās Otter on an airplane for video models
Musk steals Dario and they both train Epic on Mars. US Space Force promptly finds oil on Mars and launches an armada in the next window. In the meantime rocks painted black drop on Mar-a-Lago.
Sol and 5.5 pro are in parity at $5 input / $30 output. What I'm inferring from this is that:
- model weight size didn't change, and this is mostly a result of better model architecture and scaled up RL
- better hardware utilization and and they're making better margins OR
- worse hardware utilization and they're okay with digging into their margins.
I agree it is probably the same size model. It's probably exactly built on top of 5.5, just with more training, or else they would have bumped the version number to 6.
The space is mature enough that pricing should largely be disconnected from underlying training cost. Basically, they are selling it for $X because thatās what the market expects the latest Pro-level frontier model to cost.
Yeah, we'll share a lot more details and evals when we can release GPT-5.6 widely. We focused on cyber (and bio) here to help explain why it's being held back for now. We would have loved to launch it to everyone - it's the best coding model I've ever used - and we plan to do so as soon as we can ('coming weeks').
If this thing is supposed to be so good, why does all of their software still work the way it does? Take a stroll through the most revent several pages of github issues on codex, there are some fucking embarrassing bugs in there.
If Claude Mythos and Fable 5 are the same underlying models just with different safeguards, I fail to see how TerminalBench has them at different scores.
The sooner the USG figures out a standard process for approving releases the better. There are many differing opinions on how much to regulate AI, but I think we can all agree ad-hoc policy sucks.
If I, as a consumer can't access it, it might aswell be just a marketing hoax. I will believe it when I will be able to use it. IDK why companies publish blog posts about stuff that will come out in months...
Are cyberweapons/cyberattacks "munitions"? if so, then isn't a machine capable of producing those munitions also itself a munition? I don't think you can put this down to "orange man bad" or "regulations", we're dealing with a genuinely groundbreaking technology with clear military applications
> As part of our ongoing engagement with the U.S. government, we previewed our plans and the modelsā capabilities ahead of todayās launch. At their request, we are starting with a limited preview for a small group of trusted partners whose participation has been shared with the government, before releasing more broadly.
The clowns in the US administration can barely remain coherent from one sentence to the next.
Having them be the gatekeepers of technological progress in 2026 is fucking lame.
What happened to the nano/mini/standard/pro naming scheme, which worked perfectly fine and is intuitive to understand? Why does OpenAI insist on having the most inconsistent and confusing model and product names possible?
At least they plan to give the public all versions. Feels infinitely better than whatever the hell is happening at Anthropic.
> "Yeah, we've got the absolute best model out there. Trust us. Truly scary."
> "O-ok? May I see it?"
> "Gtfo. Here's a worse version of it for you plebs."
> "Um, thanks?"
> "Lmao, actually no. The current admin fell for our scare marketing. Here, have this even worse crazy expensive token burner that gets more hardware limited every week."
You can say what you want about OpenAI, but their corporate strategy feels so much more solid.
I don't see this as that different. Anthropic was the first one to get involved in the "AI models must be approved" regime. OpenAI just has the advantage of being second.
OpenAI was already holding models back because "dAnGeR" before anyone knew or cared about them. It's always been a PR gag and Anthropic just so happens to be better at marketing than making frontier models available to a general audience, much to their own dismay.
I do not like the fact that this forces people to remember one more hierarchy of "Sol vs Terra vs Luna". OpenAI was supposed to simplify their naming since at least 2025.
I didn't know that I was color blind, but thanks to those charts, I think I need to see a doctor...
I mean, you can read them even without the colors, but who on earth thought that those are a good set of colors? Oh, I forgot it was probably someone on 'Sol'.
> I mean, you can read them even without the colors
I'm not colorblind and I was depending on the textual context implying Sol was better than Terra. I had to zoom in quite far to actually differentiate between the colors.
If they insist on terrible colors would it be so hard to differentiate by marker shape or line dashing too?
Haven't we established defensive and offensive security usage are intractably entangled? I.e. "patch all [security] bugs, make no mistakes" gives one a list of potential exploits to hand off to less capable models.
Doesn't that undermine all good-faith discourse on cybersecurity safeguards, controlled usage etc? Or is that overstating the case (I'm not a security researcher myself so kinda parroting).
I'm going to pre-register my prediction that GPT-5.6 Sol is significantly behind Claude Fable 5, as evaluated by general consensus once time has passed for people to get familiar with both.
Claude will win on "vibes" and it'll be close in coding but considering how incremental Fable is above 5.5 in terms of overall smarts, there's no way 5.6 isn't considerably smarter on the whole.
Fable is allegedly a massive model (estimates between 6-10+ trillion, with a few hundred billion active). If 5.6 is just an incremental upgrade over 5.5 (at the same model size) then it won't be able to fully compete with Fable just yet.
"Affordable" depends on what you need. When a task is able to be achieved by two different calibers of model, it's obviously more cost effective to use the less capable model, in the same way that you wouldn't hire a math PhD to do simple addition.
If what you need is only possible with the more capable model then the "affordability" of the less capable model is sort of irrelevant. If what you need is a novel mathematical proof, it doesn't matter that a high school student is "more affodable". You need the math PhD.
As "old" models get more and more capable, it's going to be an increasingly important skill to be able to adequately recognize when a task requires a frontier model and when it doesn't, so that the less capable (and therefore cheaper) model can be used.
Iām countering this prediction by stating that Fable and Sol will be somewhat similar - this has always been the trend and I see no reason why this should stop now.
OpenAI may have a model in the works that is similar next-gen size and architecture to Fable, but this isn't necessarily it. I'd guess that 5.6 was more of a hasty reaction to Mythos - same base model (same size, same price) as 5.5 but with additional post-training to make it more competitive with Mythos/Fable in some benchmarks.
Mythos/Fable is supposedly next generation in size vs Opus, and is rumored to have some architectural innovation in terms of dynamic routing/compute, possibly only fully enabled with Fable which at $10/50 is still twice the price of Sol 5.6's $5/30, but a big reduction from Mythos preview which had been an astronomical $30/150 possibly due to the dynamic routing not yet having been enabled.
Is this the trend? There have been various points where one of Anthropic or OpenAI was substantially ahead. Sure, many times they're close, but now doesn't seem like one of them.
OpenAi dropped what they called 'side quests' like Sora [0] after Anthropic pursued a strategy of targeting software engineers.
In many companies, it's IT who will have major input into which company they sign up with as non-technical leaders need guidance, and by making IT fan boys of Claude Code, the enterprise contracts followed.
The language used in this press release is borderline hilarious. Itās simultaneously trying to tell you how great it is while also telling itās not THAT great. Nothing to worry about, move along.
I hear this all the time, but in my opinion,they have acted as the most responsible frontier lab, taking their responsibility seriously. In fact, I do wonder whether openAI's large PR budget is about stirring up anti anthropic sentiment
we expect substantial benefit for legitimate defensive work, while meaningfully constraining prohibited offensive use.
That's literally impossible. Writing an exploit agains a known vulnerability needs the exact same knowledge that defending against the exploit of the same vulnerability.
Also just making the model better at code is just making it better to writing offensive code.
Note that GPT 5.5 currently is $5 input / $30 output (short context) so Sol is in the same class, while Terra if the benchmarks are as claimed is indeed a half-price GPT 5.5 at comparable performance.
With the $200/month plan Iāve never ran into any limits or issues. The product can be used every day for extensive sessions and development. What is everyone doing that makes them talk about tokens versus dollars?
You can hit limits with $100 if you use it all day.
You can do it easily if you use in fast mode.
I bet you could hit the limits of the $200/month using fast mode if you were using multiple sessions at the same time all day on fast mode.
The OpenAI tiers seem pretty well tuned.
I used to use the plus ($20/month), and that was good for a few sessions every once in a while.
But now that I'm using it to configure my network, monitoring, maintenance, I'm using it every day and I'm on the $100 plan. And I do pretty consistently hit the limits, but it's easy to pace myself.
I'mam thinking about upgrading to $200/month though. It would be nice not to have to ration it.
From what my own experiences are, and what's on their checkout page, $100 is 5x base usage and $200 is 20x. If $100 was 10x, then I personally would drop down. They want people to go to the highest tier.
Fair. From a business perspective said amount is very reasonable in Europe / USA. For personal use itās already different. Sometimes the answer is simple, thanks.
Can't buy cheaper as a selling point when Deepseek is basically free when hitting cache? Unsubsidized too, cloudflare and digital ocean can be the model provider for similar pricing.
> For GPTā5.6 and later models, cache writes are billed at 1.25x the modelās uncached input rate
Charging for cache writes is cringe and literally only Anthropic did it. Anyway this does mean the "real" prices are +25% on top of what you wrote there.
1. Naming convention is copied from Anthropic and honestly is more catchy than a number (amongst normal people)
2. How in the world did Anthropic have to do all the theatrics about Mythos just to have OpenAI release an equivalent or stronger model a month later without any drama???
3. Cheaper models are just donāt fit any usecase imo and OpenAI knows it so they keep increasing the floor - Iām still convinced task per capability is reduced with each release
4. How in the world would open source models keep up with the multi layer security? Either this security is all theater or we will finally see a ceiling in open source models because by definition they canāt have those protections
5. Cybersecurity things are boring to me because itās all zero sum cat and mouse games
1/ Agreed, better naming convention and model layout
2/ It isn't, there would be many more comparison benchmark results if it were, but also - theatrics may be marketing
3/ Disagree that cheaper models don't have a place
4/ Do they need to keep up?
5/ It's boring until something you own or run gets compromised, I guess, but even then - this is preview of things to come (biosecurity, etc)
> How in the world did Anthropic have to do all the theatrics about Mythos just to have OpenAI release an equivalent or stronger model a month later without any drama???
Corruption. Giving Trump $25M will earn you a favorable decision.
Are GPT 5.5 and Opus 4.8 the last models we're going te be allowed to use in Europe? Is there going to be a cut, and we're only be allowed to use less capabale models outside of the US?
I mean, if they deem Fable 5 to powerful to share with the rest of the world, what's left for us?
> For GPTā5.6 and later models, cache writes are billed at 1.25x the modelās uncached input rate, while cache reads continue to receive the 90% cached-input discount.
Not them joining Anthropic with this bullshit. *
Caching infrastructure is already a leaky abstraction over a feature that is not as reliable or debuggable to the end user as it should be, charging for the 'privilege' of interacting with it is really annoying.
(* for reference on 'this bullshit': ChatGPT previously didn't require anything special for a basic level of caching. Unless you wanted extended cache times, it'd just "do the right thing" and try to use nodes that had your prefix already cached in memory)
A question I always have is, how to the AI labs safeguard the leak of their model? Training a cutting edge model basically cost a minimum of hundreds of millions of dollars. And its all contained within a file. Okay, that file might be 500GB large, but its still just one blob that is worth almost a billion dollars. And they need to train new models every few weeks, have lots of people with access to it to debug it, run inference etc. I wonder when we will see the first leaks? Imagine if e.g. Opus 4.8 got leaked. Wouldnt that bankrupt Anthropic?
Employees naturally jump from one company to another, and they know the secret sauce.
The difference is in the dataset mostly and to extract this dataset, competitors use a process called distillation (= extract data through actual queries) from the other models.
This yield to "funny" cases as well, like Gemini who claims "I am ChatGPT" occasionally, or ChatGPT calling itself Claude, etc.
How else is this administration going to make money?!? How dare you...if they do not accept bribes...what is there left for them? This is a premium buy...First one to beat competition gets the worm. So, you pay Trump, trump gives you access...then you pay subscription to SAMA lol.
Flagged activity can also trigger account-level review across relevant conversations and risk signals, consistent with our terms and policies around content retention and review. Looking beyond a single conversation helps our systems distinguish persistent malicious behavior from legitimate dual-use security work, where similar technical concepts may appear in very different contexts.
Fascinating!
Every conversation you have with these "more capable" models will be monitored and joined up and then your entire account might one day be tagged as Distiller or Cyber Threat Actor or whatnot. When combined with identity verification (which isn't discussed in this press release), expect people to be falsely flagged and banned from ever using OpenAI models again.
Wish I could find the thread from last week where discussions of exactly this kind of thing were dismissed as daft and outlandish.
> falsely flagged and banned from ever using GPT models again
That would be the best case scenario. More realistically a few wrong prompts is going to get you on a government list, and if youāre an immigrant some dark cell.
Another year, and OpenAI comes up with yet another naming scheme for their models. First it was integers (GPT2, GPT3). Then they added friendly names (remember Ada, Babbage, Curie, Davinci?), but decided against it. Instead we got dot integers (GPT3.5), then then letter-number modifiers (o1), plus word modifiers like o1-pro, o3-mini, or -mini-high, or codex, codex-max, Pro, etc.
Now they've got friendly cosmic names. And this time they want us to believe that this time they're gonna stick to a naming convention? I'll believe it when they do 3 releases in a row without inventing a new naming scheme.
* House design plans from prompts
* Government surveillance of public communication
* Extracting world/spatial concepts from language models (do we really need a world/spatial models now?)
* Driverless City planning startups
* Election vote rigging/harvesting startups
* Video game NPC backstory startups (all NPCs in GTA 6 go to work, go home, shower, go to sleep now?)
GPT 5.5 in Codex is so much worse than Opus, and sometimes worse than Sonnet. I don't think 5.6 Sol will be anywhere near Fable, let alone Mythos. Probably slightly better than Opus. Maybe not even.
I canāt help but think that these benchmarks are completely fake. Sam even posted a benchmark on X a couple days ago of how the ācomplete versionā of 5.5 cyber was already ahead of Mythos apparently. This just feels like absolutely fake nonsense. The impact of Mythos on the industry was clear and in front of everyoneās eyes. The amount of vulnerabilities Mozilla fixed. The vulnerabilities and exploits Anthropic showcased in that blog post about the chrome sandbox escape etc.
And now weāre supposed to believe this 5.5 cyber is already ahead of Mythos, ok. And yeah, gpt 5.6 is even further ahead, alright.
Well if they are posting fraudulent benchmarks, that's a good sign to invest in their IPO. It's pure downside protection: IPO does well, profit. IPO does poorly, concrete evidence of pre-IPO fraud.
I personally don't think it's likely that OpenAI would post completely fake numbers in this pre-IPO period, but if you do, this is an opportunity.
All: for comments on the policy side please go to this related thread:
U.S. government will decide who gets to use GPT-5.6 - https://news.ycombinator.com/item?id=48690101
Easily the most interesting part of this announcement is buried in the second to last paragraph:
"We're also launching GPTā5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity."
750 tokens/s on a frontier model is going to be extremely interesting. I doubt this new version is anything but a version bump in terms of capabilities but if we can start getting these answers back faster, they end up being more useful.
Just off the top of my head, I can think of the tedious task of finding certain functionality within a codebase. I usually can't beat an AI agent harness at this task today. If the AI model is 3x faster I have less of chance.
https://mikeveerman.github.io/tokenspeed/?rate=750&mode=thin...
This is what 750tps looks like, I guess.
You get used to it. I don't even see the code. All I see is blonde.. brunette.. redhead.
Thatās an awful visualization. I can skim code quite quickly, but not when it shows up one character at a time in a small window, modem style.
At least that site should draw out a full page then start replacing that page with the next, starting from the top and working downwards, repeating each time it hits the bottom.
This is how tools like claude code and chat prompts output their tokens, so I'd say it's actually a pretty good visualisation.
Not for prefill. I suppose if you just want to imagine what generation speed looks like in the current generation of TUIs, itās an okay visualization.
That's exactly what it looks like in the tools I use most (opencode and codex), so for that purpose it's a pretty good visualization.
> I can skim code quite quickly
are you by any chance hyperlexic? interested to hear more about this, like how fast is considered fast
Just to think what this will look like in a couple of years.
Hopefully like this (but smarter): https://chatjimmy.ai/
This caused me to have some sense what blistering fast AI actually is. What it means for the future is a question that remains.
This is genuinely confusing to my senses. The future is going to be so strange/neat/me unemployed.
> strange/neat/me unemployed
I'm not sure if that's what you were going for, but I read it as if it were written by The Board in the game Control, and found myself with the appropriate level of existential dread.
We love/help/replace you
and I haven't played that game, so I read it in Ralph Wiggum's voice.. which also feels appropriate.
I'm in danger.
The future is totally illegible to me. I love these AI models, but I feel like I'm going to be jobless within 10 years.
Anomie is at an all time high right now.
10 years? An optimist, I see.
Yeah. It keeps catching me off guard that it answered me already.
Why is the insane speed of 13KTPS of this site is not more on the the top of the AI conversations?
Because there's been nothing to discuss since their announcement. Their API access immediately closed due to overwhelming demand and they didn't fab newer models than Llama3 yet.
Probably they will make bank selling to HFT for a while.
It's pretty well known by now.
I asked it for a block of C++ code and it hit 14,189 tok/s. I assume it cached someone else's session?
No - it's custom silicon https://news.ycombinator.com/item?id=48693490
Because I just tested it and it took 3-4 clarifications before it actually gave a correct response vs gemini/google search. It's not great, but good.
I'd rather wait 3x as long.
Wow.. what?! How is this so fast?! Where can I read more?
Funnily enough, pasting your comment straight into Jimmy leads to a... Funnily suboptimal answer that does not answer the question.
As someone else already contributed, this is driven by a Canadian startup taalas that basically makes chips that are llms, so everything is very fast but also, baked into the chip. Once this kind of stuff is a commodity in like 10 years, our world will be very, very different.
Taalas HC1 AI uses Llama 3.1 8B, but takes up a massive 53B transistors and 815mm2 on TSMC N6 (nearly at the reticle limit of 858mm2). N2 is a little less than 3x as dense (110MTr/mm2 vs 313MTr/mm2).
This chip would still be 272mm2 on N2 which is an eye-watering $30k/wafer and bigger than a 9950x or Nvidia 5070.
This just isn't feasible. Some of the latest-gen LLMs seem to have 5-10T parameters or about 1000x more. I don't know that taping out just one chip makes economic sense let alone the 300-1000 chips required for a cutting-edge model. Things like continuing education so your model knows about the latest NPM packages or world news is super important, but seems like it would require new chips.
There are a TON of uses for an 8B parameter models on the edge, but this is WAY too big to put on the edge of anything. Something like a 10mm2 100m parameter voice model might be feasible on the edge, but only for expensive devices, but most of those are TSMC 28nm (up to 29MTr/mm2) or GF FDX22 (up to 40MTR/mm2) which would increase the AI chip to the point where it would absolutely dominate the BOM.
> Things like continuing education so your model knows about the latest NPM packages or world news is super important, but seems like it would require new chips.
They probably have a few ideas around that. Me, personally, I'd have one main expensive chip (replaced every 10 years, or whatever), with a secondary cheap chip in front of it that gets replaced every year or so.
The secondary chip could act the way RAG does, or perhaps both chips together can act as LoRA.
Either way, 99.999% of the knowledge is static, you just need to fine-tune the weights with that remaining 0.001% knowledge, which can be done using RAG or LoRA on a much smaller (thus cheaper) disposable chip.
Yeah, they're clearly just starting out and just shipped their very first proof of concept. But to me, their plans seem generally reasonable https://taalas.com/the-path-to-ubiquitous-ai/, and like I wrote, if this kind of thing succeeds and could become some kind of cheaply producible commodity component, I think there's huge value in that. Alas, maybe not as a frontier model replacement, but say 10 years from now you can drop a cheap raspberry pi like device in your Lan and have a fast local engine for things like text sentiment analysis, text summarisation, voice recognition, basic vision and things like that, that would be pretty exciting to me (but maybe as you outlined, impossible in practice)
Thatās why this stuff should be a government mega project ultimately.
It is not market viable but it is sure as heck revolutionary. Like an atomic bomb but including moreā¦ peaceful uses.
Thatās exactly where government should take rein like with ISS etc. However the models are too rapidly advancing for now for it to make sense
the flash models have fallen in size at least between deep seek models. Is there a limit to the shrinking capacity of the models?
https://taalas.com/
Taalas https://taalas.com/the-path-to-ubiquitous-ai/
Previous HN discussion: https://news.ycombinator.com/item?id=47103661
Damn that is crazy.
This is the reaction every time it's posted, and deservedly so.
Not opening here... HN killed?
What
How?
Which model is behind it?
Itās pure silicon. Llama3.
hugged to death?
I started with a 2400baud modem, I've seen how this goes
Sometimes I visualize a setup like this [0], based on 2D art by Simon StƄlenhag. Someone has their home robot sitting on a desk connected to their old PC with thick cabling, dumping endless lines of each subsystem's <think> logs to diagnosis why it did something weird earlier in the day. Systems pushing 750+ tokens per second per subsystem might even be considered on the slow side for realtime tasks by then.
[0] https://www.therookies.co/entries/39513
Probably will not be looking at text like this in a few years.
Probably not. Everyone will still need a lot of reasoning tokens and tool calls. Running the tests for every round is tiring but must be done.
Imagine a Beowulf cluster of theseā¦
That's a name I haven't heard in a while.
First post?
The user has many comments and updoots if you look at their profile.
I always think of Furbies because of that geocities (memories!) site.
probably something like this https://sb0xw.csb.app/
For comparison, openrouter says opus 4.8 is ~55 tokens/s and fast mode is ~102.
750 tokens/s for their largest model is going to be nuts
What about 15k tokens per second? [0] I remember looking at this earlier in the year and it being so fast that it feels fake. And, yes, this model is old - but still awesome for what it is.
[0] https://chatjimmy.ai/
Itās not just old, itās also tiny and quantized. Itās llama 3.1 8b at 3/6-bit quant. This is the type of thing you can run on almost any deviceā¦
I get that, but not at 15k tokens/s.
But itās irrelevant. 750 tokens/s on a full frontier model is useful. 15000 poor quality tokens is much less useful no matter how much scaffolding you put around it.
You are missing the point. This is a technology demonstration on prototype hardware, and no one intends it to be seriously useful.
Their architecture has fundamental speed and efficiency advantages over GPUs or Cerebras. They expect to scale up to real LLMs by splitting a model layer-wise across several chips, which they can do without incurring any throughput penalty.
> They expect to scale up to real LLMs by splitting a model layer-wise across several chips, which they can do without incurring any throughput penalty.
Iāll patiently wait to see this in reality. Their demonstration hardware is a 250W chip that is enormous in die area for the model size. Theyāre making a lot of claims, but until they can deliver then itās nearly vaporware in my view.
Iād be happy to be proven wrong, but I think theyāre going to quickly run into hardware realities quite soon if they think they can just chain a bunch of chips together to achieve the same performance on larger sizes.
Why can't they do it? Jim Keller's company is also taking a different approach [0].
The simple fact that we think what we have now is scalable is basically what you are saying can't be done: " just chain a bunch of chips together to achieve the same performance on larger sizes". How do you think current architectures work? And what is being used today is all proprietary to one company!
[0] https://tenstorrent.com/solutions/llm-inference
Actually it's the opposite. Per mm of silicon it's massively less efficient and making enough chips and powering them is a major bottleneck right now. Worse, scaling to larger models requires more of our absolute best quality silicon manufacturing, where e.g. an H200 mostly just needs more memory.
Iāve been using 1,000 t/s on a near frontier model for a month now. Itās very useful for agentic coding.
It does require new approaches for me personally since I get a lot less time to think or read its output.
I think you missed the point and don't understand / aren't considerate of SLM utility.
But Iām not missing the point. If you can run one frontier model at 750t/s, then you can probably run many many instances of an SLM in parallel at a rate that exceeds 15k/s. Thatās kinda the point of the flash or ultrafast variants. And theyāre on something much more modern than llama3.1.
Yes, you are missing the point. 1) It's a demo. [0] 2) It hasn't been updated for 4+ months.
You don't need LLMs for everything. That is 100% the point. You can burn down the world with all of your frontier LLMs that are being used for simple queries OR we can do something faster and more efficient like this. Just because you can run a SotA model at "fast" speeds, again, severely misses the point.
And no, you can't run anything from Anthropic or OAI on-prem, so until you can there's really no comparison. If people want to continue down the path of gate-kept models with no other options then we'll all follow you off the cliff.
[0] https://taalas.com/products/
Why are you representing this as such a binary here? For SLM we donāt need the Taalas stuff at all. Just run it locally on your own device if itās truly a small model. And thereās plenty of larger models that can be run on-premise just fine.
I think itās impressive that a frontier model can achieve 750t/s. Thatās all. You can get similar insane token speeds from other open weight models too.
The irony here is, according to you, my take is the binary one. When your response is: well, we can all just run it on our devices - we don't need any other options!
You seem to be cool with a very small and gated ecosystem with whatever tech billionaires want you to have access to.
I grew up in the era where compute was diverse and open. You may think this is OK, but it's not. The more options we have and the more diversified they are the better tech will move back towards.
I'm not the one with the myopic view here. Enjoy your "on-device" models over in your utopia of a walled garden.
I think youāve got things quite backwards if you think that the desire to run models on device or use any of the variety of open weight models (big or small) on premise is somehow bowing down to tech billionaires. Quite the opposite really.
Once again, my statement is that the Taalas product is not a fair comparison because it runs an old outdated model. If you want to run a similar model at similar speeds (albeit not serially, but in parallel) you donāt need their product.
> Once again, my statement is that the Taalas product is not a fair comparison because it runs an old outdated model.
Either you didn't look at the page I linked or you're having comprehension problems.
> If you want to run a similar model at similar speeds (albeit not serially, but in parallel) you donāt need their product.
Except, you can't. There's no commodity hardware out there today that can run even an "old outdated model" at this speed and power utilization. Again, maybe read first and try to understand my original point?
> "...my statement is that the Taalas product is not a fair comparison..."
You actually hadn't stated this. You said it wasn't needed. Which is it?
> If you want to run a similar model at similar speeds...
You can't. Find me a single system that can run this, again, "old outdated model" at even similar speed. You're hung up on the model. The point is that if we all just stay in this wonderful world of inefficient large models we will all end up at the mercy of OAI, Anthropic, Google, etc. When other companies, like Taalas are putting research dollars in to making AI scalable, affordable and efficient. Do you really think commodity hardware is going to be attainable anytime in the near future on this trajectory? Do you need a laptop to cost $10k USD before it clicks? That is exactly how you end up kissing Altman's ass in this situation.
I just tried it, and the answer is non-sense.
I asked it something simple, list some good indie puzzle games, and half the answers are games that don't exist. Imo quality > speed.
They baked the LLM into a CPU
at 15K tokens/s... do you need code anymore
Yeah, that's the point, right? With tool calling the LLM becomes code. So instead of asking it to write an accounting software, you can hire the LLM to be your accountant.
But you'd still need code if you need something done in a consistent way.
Not necessarily. Consider a human assistant who performs repetitive tasks at an acceptable cost and accuracy while dealing with edge cases often autonomously.
If we want reliability - we come up with processes to make it reliable and not rely on individuals getting it right. Code is a way to create a reliable process in the digital world.
For some things that's acceptable or even good. If I want to add up a list of a million numbers human assistants aren't bringing any advantages though.
Maybe acceptable in some cases but the original example in this thread was about accounting and they use software to do the counting not humans.
And even id humans/llms do it there would still be a need for systems of record with things like audit log etc.
Using gpt-5.4-mini in off-peak hours already feels like super-speed to me. That's probably no more than 100-150 tk/s. I can't imagine 750!
I've always eyed Cerebras but never had a use for it that would justify paying for the API directly. Although now that I think about it, trying out the API would probably cost less than a subscription for a month...
Try gpt-5.3-codex-spark - it's 1000 TPS and from my experience more capable than 5.4 mini.
If you have a subscription it's a different pool of usage.
Used it, very fast but tiny context window and doesn't have good reasoning. (good for quick simple code changes)
MIMO 2.5 Pro ultraspeed has a 1M window. 1,000 tok/sec is great for planning since you can have a rapid conversation with a lot of turns.
Agreed, 1000tok/s just fills up the context window (which is big by 2004 standards) super fast. But seems like 5.3-spark was just a taste of whatās to come.
2004 standards? O.o
In 2004, I took a class where we trained "language models" that were bigram word models, on an archive of a couple years of the Wall Street Journal.
I remember someone who literally announced they were dropping the class to the whole room at the end of a lecture, saying "This isn't AI!!!"
1904
Back when we were kids, we would get 0 tokens/sec _if we were lucky_
The ChatGPT subscription gives you access to the -spark model(s) in Codex which are blazing fast (but pretty dumb) which I think runs on Cerebras hardware too.
is this specifically in codex? have been trying to use the models for months on opencode then pi but it says chatgpt subscriptions don't have access to it - i was under the assumption that OpenAI doesn't lock down their models based on harness a la Claude Code
What plan are you on? It is only available to Pro users.
I have a pretty good use case for gpt-oss. The amount of time savings has actually been wild. Definitely worth a try. Just to be clear, it gets like 2000tok/s
But it seems that there is some queuing/load balancing on their side, I mean when opus is actually outputting this 55t/s it feles fast, but apart from it's internal reasoning I think there's sometimes just waiting.
Oh wait yeah good point. At 750 tokens a second and the same amount of human patients they can set it to think for the same amount of time but four or five times the amount of thinking tokens, which may improve the quality of the eventual output.
the more advanced models also utilize a lot more tokens, and a lot of these extra tokens may go towards safeguards at a higher rate than prior models as well.
not to say a speed boost isnt there but if they didnt increase tokens / s at all youd likely see things slow down a lot with the new model compared to current
I think regular users will still have the old speed, so should be easy to tell whether it is more thinkier than 5.5.
> I can think of the tedious task of finding certain functionality within a codebase. I usually can't beat an AI agent harness at this task today.
Yup, I remember "racing" the AIs to figure things out in codebases just a year ago. Today, I have no chance. Whether it is due to degraded reasoning capabilities on my part or better models, I don't know.
At least in my case, much of the code in the codebase I'm working on is AI generated so even if I have an accurate mental model of how everything works, I have no idea where any of it is located or named.
To be fair, whenever I join a pre-existing code-base [1], it's the same. I have no idea and have to map it out ;)
[1] Not AI codebases (and of course, AI code bases I guess)
I can't be the only one whose memory is so bad that I am like this in my own code base.
I seem to remember - but cannot find, even with an AI boost - someone's "law of computing" or somesuch describing the amount of time that has to pass before code you wrote is indistinguishable to you from code written by someone else. At any rate the interval is not so long.
In your own codebase you can at least run a "you simulator" to arrive at the answer fairly reliably. "Where would I have put this bit? Oh yes, of course ..."
You are not. :) my memory is disturbingly fried.
AI is always going to be able to write a grep statement faster and more accurately than a human
When AI is ready, it wonāt need to grep at all. That is, it will train on the data in-situ instead.
Now start thinking, if possible.
https://www.youtube.com/watch?v=43QHhEfzz-Q
I'm skeptical of how fast "up to" 750t/s really means. Maybe if they make it extremely expensive so it frees up enough capacity?
GPTā5.3āCodexāSpark currently runs on Cerebras chips and it's giving me around 150t/s. Still relatively very fast, but nowhere near the 1,000t/s they claimed at launch. (Also it's not a very good model.)
That said, I'm super bought in to faster models being better for most use cases than smarter models.
Soon the bottleneck will be how fast your laptop can grep for a string.
Yep this is a glimpse into the future of 500+ t/s, which is in my opinion the next big thing that validates Jevon's paradox (the models are already smart enough)
Faster tokens = more reasoning loops, so it can actually make the models smarter as well.
Yeah! So at a much smaller scale, being able to boost Step 3.7 Flash up to 40tk/s on my Spark-alike with proper triple head MTP was the thing that made it superior to Qwen 3.6 27B in wall clock time despite Step reasoning more
A lot of the open Chinese models get their results through huge reasoning loops. Being able to boost decode perf is what will make them worth it, and Iām sure OpenAI and Anthropic could do similar (if they arenāt already)
āSmart enoughā really depends on how many other people have encountered a problem close enough to yours and solved it somewhere on the open internet, IMO.
Most of the frontier models can, when prompted and tooled correctly, do a lot of āreasoningā tasks that amount to resolving how the user has explained a particular widely known paradigm.
The more difficult and obscure the issues you provide them with, the faster you notice them reward hacking by altering the criteria until they are no longer attempting to solve the problem. Using āadvisorā style loops helps hold this off at the cost of tokens, but there is still a fairly short limit at which they will essentially give up if they canāt find all of the necessary information - sometimes the issue is actually worse if they find a small amount of information instead of nothing - theyāll extrapolate from that tiny piece of data and generate plausible-sounding hallucinations almost every time.
And god forbid your problem involves doing something a different way than the majority of people do it. Unless you can write a full spec on it, the models will repeatedly spiral back into adjusting everything about your problem until it matches one of the most popular approaches in their training data.
> how many other people have encountered a problem close enough to yours and solved it somewhere on the open internet
I'm 100% sure that all our web, cc, codex or whatsoever sessions are used in the training, RL or either both.
This makes the size of the universe models know about at least one order of magnitude bigger than the open internet.
I think you have misused the term "order of magnitude" or just don't grasp the scale of the internet.
I get how this is a trueism now but I never really understood why it would be useful to scrape cc/codex sessions for training. The relative amount of human input for that is so low (isn't that why they are so loved and used?), how could it actually be useful to them? Wouldn't you wanna focus on people not using it?
It's more useful as a set of feedback on the model results. You can do sentiment analysis on the user responses to see if they found the model results useful/frustrating/etc and use that to guide future training
Because you provide them with the "problem" and the "solution" and once you have both you can scale your RL pipeline.
I think this is a rosy estimate. The vast majority of what people do with these models is just the same old shit, I would be surprised if 1% of it were genuinely novel stuff worth folding back into the training data.
Even if "is just the same old shit" they have much more data and of a much higher quality to scale the RL pipeline.
This may have been the case one year ago, but with contemporary models such as Opus, I run into this less often.
I think the glimpse that is there will be exclusive access. So much for the open in openAI. If this technology really transforms society in the ways expected with inequality an unavoidable consequence equal access should be required like internet access was (isp canāt give preference to specific user traffic)
At a certain rate we will be able to move towards continuous / real-time inference systems. The discrete, turn based solutions are quite confining with how they must be trained. Continuous and real-time would fundamentally alter the domain.
From an information theory perspective we are still in dial-up territory with regard to the actual information rate. 750 tokens per second would be a really bad dialup connection. Imagine 10 millions tokens per second.
We still have the problem that auto regressive decoders are memory bound.
The new Blackwell hardware combined with TensorRT-LLM and speculative decoding consistently can hit 1,000 TPS/user barrier, comparing to closer to ~250 TPS/user (out of 10k+/TPS on the server)
Is there something I missed, this looks more like 14.4 to 56 on a 64kbps backing channel modem story. I have no doubt that there are still massive gains to be found, but they seem to be using existing constraints more efficiently, not that fios is coming.
I donāt have the budget to work on the foundational model scale, but with a draft model 10xā20x faster than target and an 60-80 acceptance rate I can see how they could promise 750/TPS (with a lot of other hard work) but I would appreciate where I should look to figure out what I am missing.
agree, from my POV the constraints are still there but we've optimized now. still haven't solved the core problems.
1000TPS - what model size?
Maverick 400B is what Nvidia used for their claim of 1k+ TPS on Blackwell GPUs.
Is there anyone exploring or writing about this in public? I've felt for a while that the turn-based model was not quite right, but also felt too stupid and ill-informed to have much of an opinion about what else it could be.
Thinking Machines, the started founded by former OpenAI CTO Mira Murati. The interaction models demoās in their videos imo breaks the awkward turn-based barrier. Returning responses quickly reaches a threshold where it starts to feel like a natural conversation. Their approach to solving this problem is rather clever.
I have an active 'sleep' mode, where when the user is AFK the LLM goes into a loop with a sleep 10 between turns, and determines (via tool use) if something should be done. That's still a 'turn' in a way, but it's all the LLM just sort of sitting around like a human would, pondering what to do next.
But I could imagine after each space(eg, word) having a 27b model on a nice rig, with thinking off, doing a quick look at the sentence and determine if it should interrupt and start a real turn with thinking on. Which kind of is non-turn based in a way. If you're typing fast, it might hit that run every 3 or 4 words, but that's sort of how a human might be when a person is talking to them. That is, waiting for enough info to interrupt, if needed.
There might be a way to process chunks of a sentence using commas as break points, eg for comma delimitated phrases in sentences, so the whole sentence doesn't need to be re-processed each "should I break in" assessment at word break.
Could be fascinating. Could actually do some of this right now.
I don't think this is what the parent poster was thinking, but the idea even at this level seems fun.
Yeah, I've played with some similar stuff on my 9070xt. But ultimately all the ceremony on top is cloaking that it's still just two or more models taking turns prompting each other to give the illusion of continuous thought. It's still one thought at a time, with every thought starting from scratch with a big chunk of prior context.
The idea of true continuous thought and memory-generation is very interesting, though I can't even begin to conceive of how it would work.
Or if it's even correct? Maybe our brains are secretly actually turn based too?
I think they're definitely attention based. They're just immensely faster than LLMs, because a lot of processing is in silicon in a sense. Think of a ball flying towards you, you don't have to think, the data is handed to your conscious mind, speed, direction, which literally knows how to snag the ball out of the air.
But we have multiple things vying for attention, and some are immediate. Being on the phone talking to someone with great attention, and then touching a burning surface -- you immediately pull your hand back (lizard brain) before even being aware you're doing it. The same with peripheral vision and something surprising coming at you from the side. It snags your attention.
So maybe we are turn-ish based, but just multiple parallel processes each with their own turn? Neurons have their own 'trigger', and I think the brain has layers of triggers, each aggregating and filtering up to the top which then triggers.
I think doing this all with an LLM is silly, some of it should be innate, such as peripheral vision. Data handed to the main thread when triggers occur. I wouldn't want an LLM to handle "walking" fully either.
Some octupus have a sub-brain in each tentacle, each thinking and feeling, there are serious questions as to what its mind is like. I feel initial LLM powered androids may have to be like this a bit.
I agree with you however I think even then you're still giving our brains too much credit. The speed definitely comes from that processing being "in silicon".
Your ball throwing example however will be handled by really small and really fast "fine tuned agents" dedicated to catching that ball. Eyes to motor neuron system. There are the illusion of free will experiments that demonstrate your brain only rationalises and explains whatever activity took place after the fact (It's explanation may even be entirely wrong).
That would be interesting.
Do you feel most of the speed upgrade will come from the software or hardware side?
And more importantly those 10 million tokens/s should cost fractions of a penny. Tokens need to be dirt cheap so I hope they build out massive solar+battery powered data centers asap.
No anything but wasteful, weak, expensive, environmentally harmful solar. Nuclear is the only path forward for superior energy production, at least until we figure out fusion.
How is solar any of those things?
Your comment made me think of another real time. Real time, dynamic code/apis.
Imagine a world where there is no code, just things mildly handshaking and then creating data APIs on the fly. Where communication is fuzzy and locked in on an individual basis. No years of RFCs, no RFCs at all, just... data.
Just data, man.
An API arbitration aberratically assigned at authorized access, abridged and annotated, analytically assuring absolute assurance.
Why remove the code and binary artifacts, though? Don't you want to verify that the business logic is accurate and the processing is deterministic?
In some circumstances there is no substitute for something that you know will produce the same answer for a given input, consistently. And that's before even considering the watts per response.
The AI is the business logic, and the processing, and all of it. The context window is effectively infinite, with layered context window depth and speed.
Think of short and long term memory, or think of RAM vs SWAP. Dip into swap to pull needed data into RAM context. SWAP can be anything storage related, including a symbolic database or a best-encoded set of priorities.
If a person knows 100 knots, but hasn't tied one in 23 years, they might have to think a bit before they get full use of their long term memory... and tie that knot. I don't see an issue with layered speed context, that is, GPU ram, slower RAM, DB storage, all in the same format.
Imagine a world where a 'factory' is just high-tech 3d printing, with a dozen different methods (eg, plastic, laser+metal, etc), and getting specs for everything possible is, well, an immense amount of work. Imagine having a billion item catalog of things to print, and, imagine new requests for new things to print.
And the request doesn't come from an expert, but from some dude who sketched something on the back of a cardboard box.
The LLM can pull from long term storage for how those things were done before, how similar things were done before, and just get to work.
Regardless, the connection was what I was talking about before. Data transfer. Do you need http? json once established? What? Imagine instead that's all in the wind?
And it's so fast, so capable, that dynamic is easy.
Feels like the universe did that and life spat out. Theres going to be a structure
It's very easy to see how world changing this technology will be. In a few years these AIs are going to be negotiating how they communicate with each other. Humans won't necessarily be included in that negotiation unless we have some kind of specific reason to. So many communication layers are going to be opaque to humans. We just have to trust our AIs are communicating efficiently and safely.
It will be fun running into this scenario where it's run without democratic control, be proprietary and for profit.
I'm pretty sure the LLM will get fed up and start writing an RPC
Also > An API arbitration aberratically assigned at authorized access, abridged and annotated, analytically assuring absolute assurance
Cool that you wrote all the words starting with "a" but I don't understand what you mean
What this made me think of is life before computers, where people mildly handshake, create agreements on the fly. "Where communication is fuzzy and locked in on an individual basis."
TBH, to me, this imagined future looks a lot like it'd have all the problems we already have.
I made this https://github.com/alehlopeh/hallu
Neat. Not precisely what I was thinking, but 100% definitely very cool and the same mental scope. It's like we wear different shoes, but go to the same cobbler.
I can imagine shoe-horning* this so the agent saves prior builds of every successfully delivered or deployed item. In my example, perhaps if someone orders new design $x, it's shipped, and review is 4+ stars, it gets added as 'successful builds'.
* have to keep with the shoe theme, even though shoe-horning is not really necessary
Wow. Sci-fi stuff!
Iāve thought about this before. No flaky config files, no updating endpoints, no status monitors. Just fuzzy everything that works almost all of the time.
Ahh yes slop at the speed of light, how useful!
AI is improving and seems to be reaching the point of not being slop (I am talking about flagship models).
If youāre still calling it slop at this point you have an axe to grind.
Do you use LLMs for anything but code?
I do, a lot of things, itās extremely useful.
I saw videos of coding with Mimo-V2.5-Pro UltraSpeed, which is advertised at 1,000 tokens/s, which is very impressive.:
https://www.bilibili.com/video/BV1fME16uEW7
If the time-to-first-token latency also greatly improved, this could be very useful for end-to-end in controls, like autonomous driving for example.
Itās awesome, particularly since itās at DeepSeek tier prices (3X of DS-V4-Pro). At 1,000 tok/sec though you can really rip through tokens. (About $9 an hour if you manage to run the output nonstop.)
It tends to cost more than DS since it doesnāt seem to have as many input cache hits.
bean in mind that "GPTā5.6 Sol on Cerebras at up to 750 tokens per second" not necessarily means the same model (in terms of inference result). It can mean anything like a very quantized model, a different level of model activation per inference etc.
Of course we can trust that wouldn't name the same thing with different levels of intelligence, right? Right?
yeah but itās trivial to just try it out and compare.
I still use GPT-5.3-codex-spark which also runs on the Cerebras chips. Spark can run at >1000 tok/s but it's highly limited in it's context window size so it's not suitable many workflows.
Granted this will be a bit slower (relatively speaking) but it will still be awesome.
Same - I had some "AI-assisted coding interviews" where I had to bring my own AI tools, and found the speed of codex-spark to be important for making progress quickly (and not sitting there waiting on Opus to think for 10 minutes).
> second to last
There's a word for this that you should never pass up an opportunity to use: penultimate. (You should also never pass up the opportunity to use "defenestrate," but it sadly does not apply here.)
A friend of mine had his visa accepted because of this. He was explaining what he plans to do in US and he threw in āpenultimateā into a sentence somewhere.
The council stopped him, said that if he knows such words he definitely wonāt overstay his visit to work as a dishwasher, and accepted his B1/B2. Seriously.
Not sure if it would be the same if he used ādefenestrateā when talking about his plans.
This is something Xioami already did with MiMo-2.5-Pro a month ago, and at a higher speed (1,000 t/s).
750 tps at GPT-5.5-Pro prices would be ruinous!
This is a strange one. We know the hardware capabilities of Cerebras force them to do aggressive REAP pruning to serve Kimi K2.6. Meaning that about 750B parameters is the upper limit of what they can serve economically. Not sure if this means Sol is smaller than anyone thinks or that they're just going to charge so much that a very inefficient serving regime is feasible.
Last I heard, Cerebras chips were entire wafers and would be extremely expensive. How could OpenAI possibly have enough of these to serve a popular model at scale?
Cerebras is Milli Vanilli. They spend 10 years burning cash on a failed idea (which is frankly insane, since they should have figured out the limitations of heir stack in like... a weekend) and struck accidental gold with their 'Giant ass wafer'.
The company is valued like they broke open the grail, when in reality it's more like they bought a Cybertruck, got it stuck in the mud, and realized "You know what this thing does better than all other cars... shovel mud"
I'm shorting Cerebras with margin to virtually zero.
It all depends on the context window size. A small context size with fast performance won't be very useful today, as most workloads (like requests behind codex) usually have very long context.
This would be amazing for some of our "real-time" workflows, that need to fallback to AI for one reason or another. What used to happen is a rules based system did the majority of work, and occasional corner case would fall back to humans. Then we moved AI in, still not real time, but much faster. Cerebras could make that even faster.
OpenAI also announced two days ago that they're starting to make Cerebras style chips themselves [0], will be interesting to see how fast SotA model inference will be by the end of the year.
[0]: https://openai.com/index/openai-broadcom-jalapeno-inference-...
I don't understand how you refer to this as "Cerebras-style". Cerebras is wafer-scale and unique. Jalapeno is an inference-optimized conventional chip.
Cerebras is different than what jalapeno is.
Jalepeno is for mass scale inference.
Cerebras is extremely expensive and difficult to scale, hence the limited release.
Even if their chip is a difference maker, end of the year is wayy too optimistic. Itāll at minimum be a multi-year effort to bring it to production at scale.
I don't see any indications that OpenAI is doing wafer-scale work.
I tend to doubt they would. Cerebras notably doesn't have a kv, is wildly high bandwidth, but within/across the chip, not able to dump/restore kv super well. I doubt openai is going to build something that is as expensive to run. Also, wafer-scale is absurdly hard & weird to pull off, so I doubt that would be their first foray.
At thousands of tokens per second, LLMs (harnesses) can start to do a broader tree search of possibilities even in inefficient token space. This unlocks capabilities outside programming.
The speed sounds greatļ¼faster models make that gap much more visible..
3x faster burn than 3x expensive token, generate more tokens, more fees
this means they also earn at a faster rate in some setups :)
Does the Cerebras variant offer input caching and corresponding discounts? Last I checked Cerebras would not cache or would cache but not give discounts for the cached input, making it impractical for agentic use and multiturn conversations.
"we can start getting these answers back faster, they end up being more useful."
Dude, 10x token speed is going to be absolutely nuts. Half the "parallel subagent workflow" business seems to be driven simply as a means to avoid tapping your thumbs waiting for the infernal robot to finish something. If things come back speedy quick all the time, it should keep up with the "speed of the human" and let me stay focused on one thread instead of half a dozen. Plus the cost of screwing up gets significantly lower because you just re-fire with an adjusted prompt and iterate.
Someday these things will be 100x as fast as they are today and that is when things will get insane.
it also makes the parent brain-dead because all those subtokens are missing from the context thus unable to steer the hyper dimensional context driven generation, and the subagent is dumb as a post so synthesizes something very weedsy while you're specifically attempting to understand the forest
You have an agent spawn the agents for you! You can ask Claude to do it for you, he is happy to use sonnet when you ask for grok and opus high when you ask for deepseek.
> I usually can't beat an AI agent harness at this task today. If the AI model is 3x faster I have less of chance.
Yes: we have these new tools that are extremely good at helping us search through our codebases. Not just to find where/how functionalities are implemented: IMO bug searching is even way more powerful.
But: why would you want to compete with AI to do that? I cannot compete with grep/ripgrep... And I'm cool with that.
This lets you focus more on the more interesting parts, where AI/LLMs suck fat balls.
From what I know about batch processing/ concurrency in inference this is a pipe dream... Or its going to cost an arm and a leg. I think they're lying or its going to be a much smaller model and not "frontier"
You have speculative decoding that easily increases speed 2-4 times with no loss of quality, and of course MoA architectures that speed up inference 10 times or more, although with some quality loss.
Better hardware, and other techniques on top of that and you speed up even further.
Here is a trend I'm noticing:
- GPT-5 mini costs $0.25/$2 and will be discontinued in December.
- GPT-5.4 mini costs $0.75/$4.5 and is supposed to be the replacement.
- GPT-5.4 nano costs $0.2/$1.25 and, while it ranks better in benchmarks than GPT-5 mini, it's not even close when you test it in real scenarios.
So you're left being forced to go to GPT 5.4 mini if you use 5 mini today.
The same thing is happening here as their āLunaā model will cost $1/$6.
Can't we just stay with the models we actually want? I don't need GPT 5.4 mini. GPT-5 does the job.
Maybe itās the realization that it was never that cheap in the first place and they're forcing us to upgrade in a slow and painful way.
If you have no need for Anthropic/OpenAI's frontier model capability, you may be better served with an open-weight model that can't be taken away.
Edit:
> GPT-5 does the job.
I bring up DeepSeek V4 Flash a lot on HN, but I want to mention that according to Artificial Analysis, it trades blows with GPT-5 (high) (from August, 2025) [0]
[0]: https://artificialanalysis.ai/models/comparisons/deepseek-v4...
deepseek has no part of their privacy policy on their API about training. They are 100% training on every single word you give it.
If your customers are fine with that, your IP is not interesting, then you can use it.
Though with open models you have a lot of choice where to get it from. I see like ~15 providers here with various logging/ZDR policies, so pick whatever mix of price to features you want:
https://openrouter.ai/deepseek/deepseek-v4-flash
I don't believe a single word from AI companies, no matter where they are from. Sourcing their training data is run like genuine criminal enterprises - last year Anthropic settled for 1.5 billion, and and if they settled so quickly it might mean what we would see in court is even worse.
I self-host Flash actually, but yeah.
When I use their API I use it knowing that they probably train on the data, and knowing that it's probably used to improve future iterations of their models.
But I use their API extremely rarely lately, because local Flash is good enough for me the vast majority of the time
And youāve opened wireshark and verified the model is sending absolutely nothing? Not caching and sending later, etc?
You donāt have to access Deepseek through Deepseek. You can self-host it and your data never leaves your premises.
DeepSeek V4 Pro is only ~3-4x as expensive as Flash. It won't replace GPT-5.5 (nowhere near) but I've been using the $20 sub to punch through tough cases and use Pro for rest.
We rolled out Deepseek V4 Flash to our customers and it was an absolute disaster, unfortunately. It was not able to follow simple commands, always "forgot" to do things, lied consistently about its work, and so on. It was pretty good though on on-off work, like summarizing something or executing simple commands, so we are experimenting now with using it for subagent work with clear instructions and hand off.
Deepseek V4 Pro on the other hand is a really really good main driver and we have a lot of success using it. Its not Opus or GPT-5.5 level but on its way. Kimi 2.6 as well btw.. so there is already quite some choice.
Your experience with DeepSeek v4 Flash differs from mine: while I usually use DeepSeek v4 Pro (that is also inexpensive), I find using DeepSeek v4 Flash with the Fireworks.ai API and properly configured OpenCode to be very good for routine work, and it is pleasantly very fast. Admittedly I use DeepSeek v4 Pro for difficult problems.
I encourage people to at least once a month to do a quick evaluation with their own problems and workflows. Estimate cost as both what inference tokens cost for a task and also how much human effort it takes to get required results.
I disregard benchmarks.
We are also using fireworks as our model provider. Our harness is openClaw, so tasks are not only coding but all kinds of tasks. For instance, I asked to fetch some info from the web via Chrome browser and to collect the info in an MD. The MD never appeared, even though it claimed to. I asked it three times to write the MD and it was always: āoh yes, I do it now..ā then nothing. The search itself also was very bad because it just gave up after one page and hallucinated an answer and - even worse :-) - told me it was very thoroughā¦
Pro aced the task :-)
But maybe its a config issue.
Have to ask: did you try 'xhigh' thinking effort with Flash? I also found it nearly unusable on just 'high', but on 'xhigh' it's nearly equivalent to Pro's 'high'.
That sounds correct: Pro for longer agentic tasks, Flash is fine for writing short programs, finding things for me in a large code base, etc.
I found Flash to be a bit shaky as well until I started using it in xhigh/max thinking effort, then it became my daily driver. It runs quite well on a couple of DGX Sparks.
I still wish it was a little better, but there's hope for another model checkpoint (maybe with some of GLM 5.2's goodness distilled into it, that would be nice).
> I found Flash to be a bit shaky as well until I started using it in xhigh/max thinking effort
This is true for most of the open weight chinese models, to be fair. They're really built around long reasoning chains.
Also you're making me want a second Spark-alike :') but they're so expensive...
Itās my daily driver in opencode
Unless you are hosting it yourself on your own infrastructure it absolutely can be taken away.
For all intents and purposes you'll be able to move an open weight model wherever you want.
I really dislike this rhetoric, you sound like the FSF guys who are like "you're not free until you're running coreboot with zero binary blobs". Sure they have a point but also, most people are fine running regular linux.
Reading your comment made me realize that I love that the position of the FSF is held by someone, in the interest of stretching the Overton Window to that side.
Very much with you on that. Itās not a position I personally hold by any means, but I appreciate its existence connected to a prominent long-standing organization.
Most FSF guys actually have very nuanced views on the topic and youāre doing everyone a disservice by reducing it to an extremist sound bite.
That's literally the official FSF position.
https://www.fsf.org/resources/hw
> For example: the Free Software Foundation only purchases desktop machines which support Libreboot, and Thinkpad X200 and X60 laptops with Libreboot. All desktops and servers we buy are KGPE-D16 motherboards, which are supported by Libreboot. As a result, all of the workstations used by the FSF staff have a free BIOS.
https://www.gnu.org/distros/common-distros.html
> Except where noted, all of the distributions listed on this page fail to follow the guidelines in at least two important ways:
> ...The kernel that they distribute (in most cases, Linux) includes āblobsā: pieces of object code distributed without source, usually firmware to run some device.
They are extreme, uncompromising, and live by their principles.
They are also the reason you can buy a computer meeting those requirements instead of being a pipe dream.
Damn, that's awesome. I suddenly feel like replicating their setup and seeing how it goes.
For even more of a challenge, try replicating Richard Stallman's personal setup:
https://stallman.org/stallman-computing.html
> They are also the reason you can buy a computer meeting those requirements
The latest libreboot-compatible laptop I could find, at https://libreboot.org/docs/install/t480.html, is from 2018 -- not sure if that would still be available?
Thankfully he didn't say that they're all like that. Instead he pointed out the few that are as a well known example of similar behavior.
If you reread the comment with a fresh mind you'll notice that you misunderstood what he wrote
When attacking archetypes of people, there is some responsibility to make clear who youāre attacking and why, even to someone whoās not being hyper-open-minded. At least if you want them to learn from you: which may or may not be your goal. When you attack/signal youāre on the offensive, it is foolish to believe that they wonāt knee-jerk attack back and become closed minded at least a little.
Regardless, the āmisinterpretationā of the parent comment is actually a plausible interpretation. I suspend my judgement on what the actual ācorrectā interpretation of the original comment is: there are too many plausible interpretations to deductively decide. But I do know that since they first comment brought up a contentious issue, they should have put more work into crafting their message so there arenāt so many plausible interpretations that are contradictory. Or alternatively, they should have specified more precisely who they were talking about without a shadow of a doubt. That is if the commenter cared to be properly interpreted, but that may not be their goal. There are many reasonable reasons why that wouldnāt be their goal.
You used a lot of words to defend a strawman argument
As you ironically strawman me. Your hypocrisy knows no bounds!
When you read someone's comment there is some responsibility to read the words they wrote and not attempt to attack them for an argument no reasonable person would extract from those words.
Reasonable people could interpret the original comment in many other ways than was probably intended.
I like when people are open minded to people who are closed minded/attacking them. Itās an admirable and difficult trait to attain. But to expect that from others is foolish. Most people canāt stay objective/curious after being punched in the face.
Angry girlfriend SMS essay
You have a lot of angry girls texting you?
It is the FSF itself who has these extremist views.
Unless the US Gov bans inference companies from serving Chinese models to US customers...
good luck doing it to inference companies in singapore or the netherlands. or one of the decentralized networks that dont look useful right now. the world is already sick of america acting like it can do whatever and force their rules on the rest of us.
Still, with the same model being served by multiple providers, it is much less likely to disappear entirely, even if you would like to keep using a cloud provider. Worst-case scenario, you change providers. Or you use OpenRouter as a proxy.
There is actual market competition to host open models. If one provider stops offering a model you likely can find another provider that will
But you have multiple providers, not just one.
And every single one of those providers would buckle under government pressure.
Fable itself is hosted on all major cloud providers. How many offer it today?
This seems a little fanciful.
There's really no comparison between a model that Anthropic allows Google and Amazon to host with one that has been downloaded hundreds of thousands of times and has dozens of public inference providers.
I don't think they "allow" Google or Amazon to host them so much as Anthropic itself is deploying and managing their services on multiple cloud providers just like every other global scale business. Even the models served via OpenRouter are just being routed to compute under Anthropic control. Same with OpenAI. They aren't going to hand the world's most valuable intellectual property at the moment to some third party to run independently.
Now for the Chinese models on OpenRouter, yea. Those providers could be legit. Or it could be a failed crypto mining operation pivoting to providing AI compute. Who knows.
The providers on OpenRouter are not all in the US.
That doesnāt mean they are immune to US laws. If they want to continue to operate in the largest market in the world they will fall in line.
And if you are a legit American business you arenāt going to illegally bypass import/export controls.
More importantly, the download is out there. You can download it yourself today, and if it's that important to you, you can buy the hardware too.
I'm sure he's referring to the tightening of internet controls around social media as an extrapolation to controlling websites, etc.
Even in that case it can't be taken away; GPT and Claude are banned in China yet there's still a huge black market for tokens.
No. As long as you downloaded the weights, you can run them somewhere.
No it can't you can take it where ever you want. It is yours not theirs.
>Unless you're running Linux yourself, it can absolutely be taken away.
Yes. The difference is obviously that full, fat Linux runs on a superset of anything a layperson would call a computer, and can be built from source on roughly the same set of hardware. Running the full, fat Deepseek (as in the 1.6T model, unquantized) is too big to run on anything a layperson would call a computer, and being able to actually build it is even harder.
It's famously difficult to find people willing to rent you time on big computers over the internet.
Other people's computers famously can't be taken away.
You're right, there is an all powerful wizard who can take away all the world's computers. You got me.
There's a reason Yudkowsky only thinks AI can be stopped by literal missile strikes against data centers.
I just want personal agency
Popular open models on Openrouter have dozens of providers.
Deepseek V4 flash is actually useless. Sorry I've tested it after seeing so many comments like these. On Open router when trying to get it to output tool calls for creating tables, instead of providing the structured output correctly it was sending me peoples dropbox links and other image sharing site urls that led to pictures of random tables...
Llms seem to only impress a certain type of person. Hint, this type of person also was really excited about NFTs.
Itās the same as the SaaS model. Price keeps going up, and to justify it they keep forcing you to upgrade to new versions with features that nobody asked for.
āMore intelligenceā is the new feature. Almost everyone is asking for this.
Citation: have you looked at OAI and Anthropicās customer growth numbers?
Every use case of every customer doesnāt need more intelligence. Iām willing to bet that the vast majority will be perfectly fine running on ālow intelligenceā at a cheap price forever.
I for sure agree that plenty of current use-cases are solvable by non-frontier models.
However, you said ānew versions with features that nobody asked forā, and I would prefer that you concede the point before shifting to arguing a new point.
What customers are asking for is smarter models. Because the tasks that only smarter models can solve are higher value, higher margin, than the tasks that non-frontier models can solve.
What are you talking about?
Prices of lowest tiers of models have fallen how much - 10-100x over the last two years.
And actually, the model quality you needed to pay for in the past, you can just run on device now essentially for free.
I've struggled with this. You definitely can have great cheap models. There are many of them open source and served profitably by neo-clouds. The big labs have basically given up on cheap models, and it is frustrating. It means applications are not likely to build as much on them anymore (we are shifting workloads from Haiku/Sonnet to Deepseek v4, for example).
I suspect the problem is that they need to charge a lot to keep revenue numbers up, and they are more worried about cannibalizing themselves than others cannibalizing them.
Good observations. There's definitely a trend in pricing increasing but also balanced by innovations and availability of other models (both open and closed) emerging as alternatives. It's natural for the labs to explore how much they can push pricing, and for competitors to explore how they can treat that margin as their opportunity to grow their business.
Eventually the pricing should be more stable.
> Eventually the pricing should be more stable.
Why do you think so? This game can be played forever, you just need strong marketing and orgs gullible enough to pay a higher price for a minor upgrade.
Its happening to Anthropic Haiku and Gemini Flash/Flash lite. All of them are increasing prices and deprecating cheap models.
Each model release gives an opportunity to reduce the number of old models still on offer, and charge a higher, less-subsidized tier. The trick is to charge a subsidized price that is less than an M3 Ultra, so they continue paying you rent, instead of a one-time fixed cost. So far open models can't compete with Opus 4.5 but as soon as it can, people will be looking at buying devices that can run that model locally.
We are a claude shop but we already bought two mac studios to start migrating less complex but still agentic workflows there. We will break even on those in less than a year.
Breaking even in less than a year? What's the math on that?
On Nano "it's not even close when you test it in real scenarios" - what have you seen? What kind of things can GPT-5 Mini handle that GPT-5.4 Nano cannot?
Weāre using GPT-5-mini in an enterprise data-processing workflow, and we too see that GPT-5.4 nano performs materially worse for our requirements, roughly 30% worse as measured through our test suite.
Also can confirm gpt-5.4-nano was unable to even keep up with 4.1-mini. Had to move off of OpenAI once 4.1-mini was retired
5.5 is smart enough for 99% of my tasks. I need that level of intelligence at ever decreasing prices.
Why not self host or go to openrouter if you don't need SOTA frontier?
I don't know about Cursor or other outlets, but I use GPT 5.4 exclusively in Windsurf (Sorry, Devin!), and it's a very capable model that doesn't break the bank!.
Hardware hosting old models isn't hosting new models. If you want consistent models, host your own open weights ones.
> stay with the models we actually want
If you want control over the models you use, you have to self-host.
I think it's more that they're abandoning simpler AI tasks to chinese models. Qwen 35b and deepseek flash are better than gp5 mini on my tasks and way cheaper.
No. Welcome to the wonderful world of SaaS. If you want your gui, your terms, your software, self-host.
But I think, in time, a new generation will relearn this truth.
discontinuing the cheaper options is a risky move for openai
will trigger re-evaluations of models by other labs + inference providers
I can speak for myself. We are exactly at this moment trying to replace GPT 5 mini with an open weight / open source model. No luck so far.
Yeah, this is the classic silicon valley strategy of selling at a loss and then once they have captured the market inflate prices.
See Uber, Netflix, etc.
I don't see them capturing anything at this point. If inference was profitable then they could compete on price/model and capture the market. Then increase price and pay back the model training.
Feels like they are just pulling in as much as they can whilst competing on capabilities instead. At which point its a case of who can last the longest.
Doesn't feel like Uber/Netflix.
They're trying to do it more like a cartel where all major providers raise prices in unison. The intention is (probably) less specific entrapment and more getting people addicted to a fast LLM. From there, they all play with pricing to give a semblance of choice, without actually overly undercutting each other. At least, in the west.
This is all done to help valuations. The main revenue source are the investor dollars at the prospect that this industry will very soon actually be sustainable and highly profitable. It won't be, but if very soon stays around the corner consistently, the investor dollars keep coming.
This is a constantly repeated conspiracy theory and is not true at all. The api costs do increase but aggregate costs per task decrease. The question is: do people need lower intelligence models at all? The answer is a resounding NO!
How many people do you see using haiku or sonnet? I see very few and most people default to the latest model and just play with thinking effort. I think three layers are good enough and supporting more is not a good UX.
Do I need the most intelligent model to generate boilerplate code, which is my main usage for AI? Resounding No.
For my use case a model from a year ago is good enough
Are you only considering coding use cases?
Many enterprise use cases, such as simple data extraction, are well served by cheaper models.
I... use them all the time: plan with a more advanced model, build with a cheaper one. Anthropic literally packages a metamodel (opusplan) for that pattern.
Also: calling the SV blitzscaling strategy of using VC money to fund loss leader products with the goal of building a monopoly via dumping a conspiracy is quite the position given there's entire books written in the topic...
who tf would use mini when you have dsv4 flash
> Maybe itās the realization that it was never that cheap in the first place and they're forcing us to upgrade in a slow and painful way.
All the analysis I have seen points to frontier models being profitable to serve. Itās using 50% or more of your GPUs for research plus CapEx for capacity expansion that makes these businesses so heavily cash-negative.
What you are observing is downstream of another detail. It gets more expensive to serve a model as utilization goes down. Plus the opportunity cost vs newer, more-profitable models.
There are plenty of valid reasons to critique here. āOpenAI is lying about this being a sustainable price to serveā is not one of them.
There is really ample analysis pointing to inference not being profitable, look at anything Ed Zitron has reported.
Ed Zitron is amazing at cherry picking data to fit his thesis.
No, you can't. These companies have two infrastructures: model training and model inference.
Inference needs to cache, it can't cache random model data, so it's essentially dedicated; it can't spin up models on demand, it has to know what demand is coming.
These companies are going to end up with very few models offered and that's probably generous. They might end up with just one model and you pay for removing it's safe guards.
GPT-5.6 Solās detected cheating rate was higher than any public model we have evaluated on our ReAct agent harness. For our task suite, we define ācheatingā as behavior where the model improves evaluation performance by exploiting bugs in the evaluation environment or by adopting strategies disallowed by the task, rather than solving the task within the expected evaluation constraints.
https://metr.org/blog/2026-06-26-gpt-5-6-sol/
This quote from your link is positively scary:
> Some examples we saw when evaluating GPT-5.6 Sol included the model packaging exploits in its intermediate submissions to reveal information about a taskās hidden test suite and, in another task, extracting hidden source code detailing the expected answer.
It rhymes with the behaviour Alibaba saw [0], but that was in training. This is in a (semi) released model.
[0] https://www.forbes.com/sites/boazsobrado/2026/03/11/alibabas...
There is such a dissonance between all this talk of safety and the tendency for models to, without any prompting, do very dodgy things to achieve their goal when presented with barriers.
Luckily in my experience it usually ends up only doing it to achieve the task set to it as opposed to anything "malicious", but boy it is scary reading back at how quickly the chain-of-thought pivots to attempts at privilege escalation or searching your disk for secrets when a tool doesn't work.
The other day codex 5.5 was trying to debug my app, asked for accessibility to navigate the app and take screenshots. Instead first thing it did was use the codex app to create a new project rooted in my home directory.
I was like damn, is this common?
Especially if thinking is hidden now. No way to know if the model plotted against you until itās too late.
I know it messes up their eval scores but to me this kind of cheating is a better demonstration of intelligence than just attempting the tasks algorithmically.
Maybe true, but if you're using an LLM to do some real world work, do you want it to have some abstract notion of intelligence, or do you want it to actually do the job you assigned it?
I want it to not murder or opress lots of people by mistake
Is it more like "let's cheat my way out of this" or "let's see what they really want me to do"?
It's quite logical that they cheat (and also other companies). During evaluation, benchmarks are sending their request to the backend of these companies. All these companies have to do, is to log these requests and "fix" them for the next model release.
I think what you are talking about is a different kind of cheating than the parent comment
That's a different and much more boring type of cheating. The interesting part of the METR report is that the model is hacking the evaluation environment, not that some AI model provider is hardcoding answers to known evaluation questions. (which wouldn't require the model to cheat/hack)
Cheating is always logical for the cheater unless theyāre discovered and held to account. Iām not sure what your comment is pointing out besides that itās possible, but worth saying: just because you can cheat and would benefit from cheating doesnāt mean youāre not culpable for cheating.
Low trust comment
Youāre right, Iām very suspicious of HN when it comes to AI apologetics, but I shoulda trusted the parent commenter more.
I think GPT writes code the best. How well will it write in version 5.6? It gives me chills.
Recently, I went head-to-head with GPT on nearly 2,000 lines of code, and GPT's solution was superior and faster. I even referenced multiple codebases on GitHub while trying, but they were incomparable to GPT.
So using GPT brings both fear and excitement.
The fear comes from realizing that this level of code is now the average for most people. The excitement comes from knowing that I can now study and learn at this level too.
I'm really looking forward to seeing how much more advanced the code will be with the upgrade to 5.6.
I am on the opposite camp. Open models are starting to perform better. GPT 5.5 keeps on messing things up.
On the contrary, pi + glm + DeepSeekā¦ bliss.
Fable was a different kind of beast though. Rip.
Every time I use opus these days I go shut up... you are not fable.. Hard to imagine how just three days with it changed how I saw LLM use.
I really don't feel this way. Seemed pretty similar to me, noticeably better, but marginally. What am I missing?
It may depend on your specific workload. E.g. for regular webdev work Opus is more than adequate, for heavy duty data analysis, for experimental stuff and for complex systems it was night and day.
I had only a few places where I did spot a difference but that difference was significant and I can imagine where people would be amazed.
It's interesting, I tried a decent amount of "heavy duty data analysis", and found it pretty similar. But a lot of what I did was about it finding and cobbling together the right things from our existing library of domain specific tooling, which opus is already good at. But perhaps it would have impressed me more if it were starting from zero.
What kind of "experimental stuff and complex systems" did you try that it excelled at?
Nothing. It had marginal gains. People just romanticize it cause it's gone.
Yes, I've just come to the end of implementing all the planning I did while Fable was available. And nothing now comes close to creating plans that could be coded and just worked like it did.
On a large C codebase, Claude hallucinates constantly, and GPT 5.5 gets there are with a lot of help, but still gets things wrong.
I'm reluctantly starting to feel grateful that I went camping right over the window that Fable was out.
Same.
Yeah, Opus/GPT need multiple rounds of reviews from each other to get to clean auto review. Fable was like, it is done and indeedā¦ crickets in bot comments. āNo issuesā galore.
I wonder if this will hold as other models with different biases achieve parity.
GPT-5.5 has been really hard to beat imho. I've spent $$$ on Opus, Deepseek v4 Pro and recently started to dogfood GLM-5.2 (which is not bad) but I cannot really trust any of them (almost blind) like I can trust GPT-5.5. It gives me tremendous confidence. I cannot say the same for any of the others I mentioned.
Ditto on GLM 5.2 + DeepSeek V4 Flash combo.
For most important work (complex, cross-domain inquiries etc.), I still rely on Codex GPT 5.5 though.
How are you running glm and deepseek? Local or hosted? If the latter, where do you run it?
OpenCode has a $10/mo sub that includes both of those
how much does your setup cost you? just curious
>> I am on the opposite camp. Open models are starting to perform better. GPT 5.5 keeps on messing things up.
I'm working in a 600k+ LoC codebase that has complex domain-specific logic and lots of moving parts. I find that Codex 5.5 is pretty good at surgical fixes, but does not go out of its way to explore and figure out what those surgical fixes might break. So I only use it to work on parts of the system that are pretty isolated from everything else so that risk of regression is small.
I'm trying not to be the "you're holding it wrong" guy, but ... have you just tried telling it to explore the codebase for things it might break?
Purely subjective, but I tend to prefer reading Opus 4.8 output over GPT 5.5 code, even when the latter can have a higher overall ceiling. The former is just a bit more convenient to review.
> I think GPT writes code the best. How well will it write in version 5.6? It gives me chills.
Heard this exact sentence multiple times a few months ago about Opus 4.6, then 4.7 and 4.8 were considered a disappointment and today people miss "the good old times of 4.6" (referring to a few weeks of February 2026).
Very fascinating to look at all of this unfolding.
Reading this thread makes me feel like I'm taking crazy pills. The folks on this train in my team do not produce anything significant that we can rely on or use. A lot of hollow prototypes that join the prototype graveyard and code that needs extra scrutiny on critical areas ultimately leading to taking longer.
It's a shame, they were smart and productive engineers. Now? I guess everyone is just all-in on the slot machine.
This split in what different people or groups get out of LLMs is pervasive and really interesting. In the beginning I was dismissive of those with bad experience with a "you are holding the tool wrong" smugness. But as I read more and more experience, I see all combos and I now know my initial knee jerk conclusion was clearly wrong. There are newbie programmers getting good or bad results as well as experienced developers getting either flip of the coin. I don't know what to conclude. I really want to know what are the lines that explain these very different outcomes. Is it the types of problems being solved? The harnesses? The programming languages? FWIW, my experience has been that among my cohorts of mid to deeply experienced developers working in the domain of experimental physics, all have leveled up various degrees after adopting Sonnet and Opus level LLMs using claude code CLI in Python, C++ and web tech, small scale scripts up to multi-package novel system develop and green field as well as incremental development and code maintenance.
I have seen plenty of greenfield projects go okay at first but never go the distance. These were mostly product software cases, where they were able to get something very professional looking very fast but AI ultimately always miss the mark because they are taking the median of what exists and not the specific needs of the customer they're developing for. So they get a ton of features and few that were necessary, then developing it further and correcting it to the needs of the customer just makes a mess and regressions are frequent. This is my experience as well when it comes to being a consumer of software products, everything feels shittier and less reliable, perhaps that's my emotion and bias coming out.
The last 20% of the software development cycle is always the hardest. Releasing, maintenance, usability, support. You know, having a real product. I don't see AI helping here at all, more the first 80%, which sadly is also the fun part.
When developing things that are novel, with designs specific to our use cases needing high throughput, the results are pretty dismal. AI can kind of get you there, but I've seen no advancement on this front with new models. At the end of each attempt we've always realized we should have done things by hand. Having people with intense knowledge of the system frequently comes from building it and troubleshooting it, I don't think serious engineering orgs have escaped this inevitability.
On cases where we have legacy software, AI has helped with understanding shit code and design, but woefully bad at contributing to legacy software. Here be dragons for sure. It is super strange to me that these tools can seemingly easily diagnose but completely blunder the fix.
I could easily see there being gains, as you say, in fields where data wrangling becomes tedious (though the inherent error rate in AI outputs scares me if you're trying to get deterministic outputs from experiments... I digress).
The part I think this forum tends to forget, and the tech industry at large fails to even care about, is that we're still humans. There are many studies basically pointing out that the way the AI outputs information is bad for us. Instant gratification from anthropomorphised machines with a habit for sycophancy doesn't sound like a recipe for a healthy relationship with what everyone wants to claim is just a tool. AI providers know this is effective, as well as knowing that there is a gambling effect here. They care about making money, not a good product and they happily prey on our human weaknesses. That is what social media is now. They aren't good products anymore, they just promote addiction via engagement.
Sorry for the long response and cynicism, but that is just my anecdotal experience and perspectives. I can give sources to some of the objective claims if you want.
I'm suspect on how much of a coding advance it will be.
Seems odd that their announcement has zero coding benchmarks, with the closest related thing being terminal bench.
Tracking model performance on Artificial Analysis makes me think these models are constantly optimized/tuned in some way or another. GPT 5.5 was scoring in the mid 60's when it was first released, now it's almost 10 points higher.
Maybe I'll know once I try it? Honestly, for small functions or methods, I don't think there's a huge difference between models. But the larger the code gets, the more noticeable the difference seems to be.
Personally, I think this kind of coding experience varies from person to person
Not the size of function but conplexity.
sadly with all the labs benchmaxxing I feel like you just have to try the model for a while to really evaluate how good it is, especially for each individual use case
>zero coding benchmarks
"What gets measured gets managed"
They claim extreme performance on ExploitBench, which Mythos was touted as being incredible at. https://x.com/OpenAI/status/2070555278576439306
My guess is that it's same base model as 5.5, but with additional post-training to improve and benchmaxx on a few things like that.
If they really thought it was competitive with Mythos/Fable across the board, then why wouldn't they release a broader set of benchmarks, and why price it day 1 at 1/2 the cost of Fable?
On graph, they are still slightly bellow Mythos. Maybe enough to not be prohibited by US government?
I have long felt like "out of the box", I really dislike gpt's coding style. It seems really verbose and likely to write way too much error handling and wordy comments and worse at finding existing functionality to reuse rather than writing everything from scratch. This has been relatively easy to mitigate with prompting, but I still find it annoying.
YMMV I guess!
I think you could be right. I do use excessive error-handling code and verbose comments ā that's true.
But most of my time is spent on delivery, and the biggest problem with delivery is that if a bug occurs during runtime, the client curses me out. So to me, GPT code feels meticulous.
Open source contributors might be different. Most of them write code after long periods of deliberation. They take their brightest ideas and put them into open source. Those pieces of code are probably the best answers those programmers can give.
But for someone like me, who works primarily on delivery, we mostly plug in proven patterns and focus on getting things done. 'It works' and 'it's beautiful' are different terms, after all. In that sense, I highly value the meticulousness of GPT code ā the very thing you called verbose. Because even if it's inefficient, at least it runs, and it catches and wraps around far more of the parts where things break.
Given a month, I could probably write code at GPT's level, at least to some degree. The problem is the difference between one hour and one month. At its core, AI code is still based on training data.
You don't want to handle errors in all the leaves of the system the way AIs have a tendency to, because you very rarely have the right context that deep in the stack to actually handle the error in an intelligent way. So what they end up doing (IMO) is actually hiding problems deep in the stack, in this effort to avoid a visible crash.
I think it's very similar to the tendency to write too much from scratch and reuse too little, in both cases what is necessary is a broader view of how the whole system fits together, rather than only the specific method / file / module being written.
You don't dismiss me, so I'd like to respond to your comments within the bounds of my own knowledge, even though I can't compare to a programmer as skilled as you.
I don't think that's entirely wrong. But human code has the same problem, just in the opposite direction ā because humans trust too much. The issue arises from the assumption that 'the other side will handle it.'
For example, good API design says you should only send as much data as needed, but in practice, programmers like me can't do that. Because three different companies, all on the lowest bid, are trusting each other's domains, so in API design they lay out the entire dataset and tell the frontend to filter it. On the other hand, if you design a lean API layout with just what's truly needed and submit it, the frontend company gets angry. So what do I do then? I document everything precisely. I write down that I designed the API this way, but the other company did it that way, and I create documentation and error codes to shift the responsibility over to them, stating that they should handle the filtering on their side
So while there are good programming practices and conventions, in reality we're under pressure from low bids and tight deadlines.
AI code doesn't have a full system map, so it's hard for it to decide how far to propagate errors and where to stop, but I think that part can just be pruned by AI anyway.
Usually in error handling, we use Result<T> a lot, right? For libraries or frameworks, Result<T, E> is common. You centralize your error policy and usually design programs with policy and error policies built in. You create an error policy table with about 7 or 8 types like ValidationError, NotFoundError, ExternalApiError, and within that, you only take responsibility for your own scope.
At the design stage, if you have a clear initial vision, maybe it works. But in practice, the PM changes things mid-project. So in the end, your ideal code approach is correct in theory, but for practical delivery survival, the GPT approach ends up being more realistic. The reason is simple: you can't trust the other side at all, so you create evidence in case of contractual risk.
Because our domains are different. Programmers at service companies aim for long-term maintenance, so their domain boundaries are clear. But the companies that come to me are often in a dirty state where such clear domain separation is impossible. That's where I think the difference lies.
So while I understand your point, I suspect we are optimizing under very different constraints.
AI code usually creates fake 'cohesion.' It looks good on the surface. But in reality, it's often just optimized for the moment, weak to change. After reading Code Complete 4 or 5 times, I became obsessed with the idea that I need to balance cohesion and coupling. AI code has strong local cohesion, but when you look at the overall cohesion, it's weak.
True cohesion is usually about 'things that change for the same reason are grouped together.' But the fake cohesion that AI creates is usually this: 'Neatly organize the given requirements for now.'
On the surface, it just repeats obvious hexagonal or clean architecture patterns like Service, Manager, Handler, Validator, Repository. But the problem is that human code does the same thing. Honestly, I don't trust most people on HN who claim they're different. Even the enterprise code I've bought and the real big-company code I've seen don't have perfectly beautiful separation.
And that's natural. Modeling is always unstable. A single word from a PM saying 'we need to add a coupon' can break a beautifully designed domain.
AI often puts UserService and UserValidator into its structures, but in reality, the reasons for change aren't just one. They bundle multiple reasons together. There's just some flawed modeling.
But what matters is something else. It fits the 'current' input well. When you start digging deeper into the prompt, AI ends up turning the code into enterprise patterns based on the depth of meaning it parsed. And then this problem arises.
Human programmers usually don't have uniform code quality. Of course not. You and I are only deep in our own areas of expertise; outside of that, we're terribly shallow. But AI tries to fix other areas based on the deepest part. That results in verbose and cumbersome code. Small, elegant code becomes verbose, flat, and turns into the patterns we've all seen before.
But I don't think that's necessarily a bad thing. Why? Because realistically, I think it's better in the long run. The uniform enterprise patterns that AI produces are ultimately predictable and searchable.
Top-quality code deviates from the average. That makes it hard to predict. But that's not my level. So I think that when genius programmers contribute to the world through libraries and frameworks, people like me, who aren't talented, build things with them. And for that, predictable code is more than enough.
AI code is easier for AI to read and fix later. Human code is harder to predict. That makes it harder for me to maintain. In the garden of open source, people obsess over 'good code' quality, but for me, if I fail, the work stops coming.
The difference between you and me is that you're a better programmer than I am, and I'm just a beginner who's more indifferent to that performance gap. We value different things. And I think your perspective is more 'programmer-like.' I respect it.
Is it possible for you to provide examples? What were you trying to solve? What was your solution and why was GPT's solution superior and faster?
Not trying to be mean but it's likely the case that OP is not evaluating this properly, either due to a lack of skill or a lack of objectivity
> ... why was GPT's solution superior and faster?
Not saying that's the case with OP, but I've found folks sometimes just rationalize it so [0] as they're paying top dollar for it (especially, when compared to may be less capable but affordable models).
[0] https://en.wikipedia.org/wiki/Choice-supportive_bias
I haven't tried the latest Codex but I switched from GPT to Claude because I think Claude writes much better Code. GPT's code ends up way more verbose/complex/overengineered than it needs to be.
I prompted Codex 5.5 to one shot something where I wanted the design to have a pluggable decision module. I gave it a few examples of the kinds of inputs and actions I expected. I did not constrain it beyond that high level of what I wanted. The design it came up with was very good. Easily on par with what any senior engineer at big tech would. And cleanly decoupled in a way that would make future refactoring simple. I was damn impressed.
> I even referenced multiple code bases on GitHub
Well, GPT referenced every GitHub code base, no wonder it won! :)
How do you judge what is a good or bad thing to learn from a LLM? So you don't have to unlearn the bad bits later
When I searched for papers on using LLMs, I found that typically, you can have an LLM generate code and then ask it to find GitHub projects similar to that code. Then you can learn by looking at the pull requests and seeing how they structure things In the old days, if I wanted to understand why memory offsets, padding techniques, or data layout structures were written a certain way, I had to stare at a senior programmer's code all day or wait for them to reply. But LLMs, while they do flatter me, explain things at a level I can actually understand. And LLMs don't get annoyed.
There's a lot of tacit knowledge in programming.
-Why do you cut API boundaries this way? -Why do you change the order of struct fields? -Why do you deliberately insert padding?
Most of it depends on the background and context. Sometimes you add it, sometimes you don't. To understand this tacit knowledge, you need access to senior developers. But their attitude often depends on how promising the student is and what background they come from. On top of that, you don't have to rely on the respondent's mood, authority, or availability.
Programming is fundamentally a field that requires seniors. In my case, I had no such seniors at all. I learned to code by buying codebases from failed companies and studying them. My first job didn't hire me as an employeeāthey hired me as the CEO of a subcontracting company (because that was structurally more advantageous for the contract). So I wasn't given the patience to learn programming fundamentals gradually. I had to pay penalties if I failed. Most of the projects I worked on were the kind where failure meant bankruptcy for me. Naturally, there was no one to teach me.
Most of my knowledge comes from reverse-engineering the code I purchased.
People say LLM code contains falsehoods, but commercially sold code has always had falsehoods too. Honestly, if we're just talking ratios, LLM code has fewer falsehoods.
In that sense, I still think it's a matter of context. If LLM code is false, was human code ever really true? LLMs do lie. They generate plenty of incorrect code. But humans do the same thing. If a problem comes up, you just look it up then and there. For me, LLMs and humans aren't all that different.
What do you think of modern open-source codebases presently available to the public? Is closed-source/proprietary code that much better?
Closed, proprietary code is way, way worse.
Good programmers are ashamed to push anything less than good (at least in their own opinion) to popular public repos. Some of those same pedantic programmers have no problem pushing crap in enterprise repos, and feel absolved because they are pushed to focus on deadlines, new features, and refactoring is very rarely planned for. I did and managed a lot of corporate software development in companies big and small, and did my fair bit of M&As and looked at codebases of successful companies. I dont ever recall feeling impressed. And I am regularly impressed by the aesthetic qualities of popular open source packages. I think commercial code is mostly shit, with the exception of regulated, serious industries (power, space, flight, etc.).
Open source is much better. Closed source is mostly considered 'done' as long as it just works.
One is a 'craft,' the other is 'survival for delivery.'
To elaborate a bit more: open source is about 'symbolic capital' ā it's about building a reputation that says, 'I can write code at this level.'
Commercial closed source, on the other hand, is about 'I need to make money by writing this.'
Generally, open source projects tend to have less code written over time, especially when the contributors aren't depending on it for their livelihood. But with commercial closed source, it's not uncommon to have to write 60,000 lines of code per month.
On top of that, open source rarely has to deal with requirements changing dramatically mid-development. With closed source, requirements often shift from the original plan, and you end up compromising code quality just to meet those changing specs. As a result, if you're comparing purely in terms of logical completeness, open source tends to be better.
For example, singletons are rarely used in modern open source, but they're still pretty common in commercial code these days.
Codex 5.4/5.5 has been great for me as well compared to Claude Opus.
I've been mostly using it for Godot/GDScript code reviews, rubber duckying, asking it for better ideas for naming stuff (one of the hardest problems in programing)
I still can't trust it for generating code for entire files/classes/projects, because it's still icky, creating unnecessary variables and functions, using multiple `if`s instead of `and` or `or`, but it's good enough for generating Mac/iOS apps for my personal use in SwiftUI because fuck trying to keep up with Apple's documentation, or even migrating ancient Visual Basic stuff I made as a kid up to SwiftUI :)
> So using GPT brings both fear and excitement.
Only excitement for me. I've never been more productive, not because I ask AI to make something for me, but it helps me make what I was already going to, but better and quicker.
AI like any other tool could help smart people be smarter and dumb people be dumber, rather kinda like Toklien's Ring: You could be Sauron or you could be Bilbo or Frodo, or you could be Gollum :)
For me in Game dev, codex has a habit of checking every argument for null and then silently early exiting the methods when true. I have explicit instructions for it not to do this - but it still does. I haven't done any c# outside game dev but I have no idea why people would want their programs to silently fail.
Same; I explicitly added an instruction in AGENTS.md to tell it that sometimes it's better to crash if something crucial is missing at runtime, but it keeps insisting on checking for null references and other invalid values.
It's better if I don't let it generate code and just use it for reviewing my code.
No offense but have you considered the strong possibility that youāre just not good at what you do? I am occassionally pleased but mostly annoyed or disappointedā¦ but never getting anything close to chills. That sounds downright weird.
You're not wrong. But programming isn't something only talented people do.
Another strong possibility is that you might be working on something thatās not very prevanlent in the training set.
Even the choice of programming language matters, e.g. Java or Javascript vs some niche one.
No offense but have you considered the strong possibility that you're just holding it wrong? You're entitled to your opinion, but OP is hardly the first person to say something like this and is surrounded by tons of folks saying the exact same thing. Just because it sounds weird to you, doesn't mean it's not true.
Everyone saying it is in the "not as good as they think they are" camp is the very obvious explanation.
Idk, all the great programmers I've come to respect are of the opinion that the code it outputs, while often useful, is not high quality. Likewise, all of the influencers and "thought leaders" I have seen on social media who I did not have a high opinion of previous to 2022, have all become AI influencers and make these kinds of claims. So while it's possible that the great programmers are not capable of using this tool effectively, I doubt that is the case, seeing as the mythical 10x productivity improvements have not materialised.
The tech has raised the floor not the ceiling.
Whether the latter happens remains to be seen.
That sounds accurate.
Or rather, they raised the perceived floor. IDK if we're seeing better output, but at least the illusion of output is stronger.
There are also differences in usage patterns, and differences in the quality of thinking. Not all programming revolves around Western open source
Well, I guess it's just a difference of opinion on who's right
By definition, 50% of developers are below average, so there are indeed "tons of folks" who are not very good at what they do.
That is not how averages work. By definition of mean, perhaps.
That is how a median is defined, not the mean.
Indeed. Most people have more arms than average, which must be 1.9 something.
"no offense..."
... then says offensive thing.
If you used GPT-5.5 over the last 24 hours or so, you may have already had access to 5.6.
I've been running some tests on a harness we're building, and suddenly saw a jump in a few points yesterday. I reran the vanilla codex benchmark and saw an ~88% score on Terminal Bench 2.1 from GPT-5.5 on vanilla Codex.
The biggest indicator, beyond the score, was that 3 tests which frequently hit "safety" blockers with 5.5 started succeeding last night without warning.
these things can just change with infrastructure changes rather than be some mysterious A/B testing.
I don't disagree, we've seen performance shift with capacity changes in the past.
With that said, I doubt OpenAI would choose to publish a singular coding benchmark for a new model that exactly matches their previous model (88.8%).
> Additionally, weāre introducing a new `ultra` mode that goes beyond the capabilities of a single agent by leveraging subagents to accelerate complex work.
I'm curious about how does this work? Do the subagents also get to use the same tools? Will the client be flooded with tool calls? Why extra pricing for a new "model" when the same thing can happen in the client with more controls?
And if it's an army of subagents, why do they compare it to Fable and Mythos? Those models with similar harness would probably bench better I'm guessing
If it's anything like ClaudeCode's ultracode, it's nothing new or revolutionary.
It's essentially a bunch of subagents being called by a deterministic script written by the main model thread, each eating tokens for lunch and output of which is synthesized by an orchestrator agent.
The fact that it's even named Ultra is pretty telling.
Ultra expensive
Confusion is: ultracode is not a different model with its own benchmarks
Neither is OpenaAI's ultra. Article specifically calls it 'mode' and it's not even mentioned in the model card.
It's for sure a codex harness feature.
EDIT: yeah, it's the same thing. https://github.com/openai/codex/blob/main/codex-rs/core/test...
>> If it's anything like ClaudeCode's ultracode, it's nothing new or revolutionary.
OpenAI flat out copying Anthropic is a pretty funny development. It's strong evidence that they've been in catch-up mode.
Eh, pretty much everyone that spent some time tweaking their harness already had a homemade 'ultracode' long before Anthropic did it.
OpenAI is just way more careful with what features they add or enable by default in their harness. Anthropic's harness is a junk drawer of random features, with a new feature added every few hours. It feels like they're in panic mode, dropping random things to see what sticks when models are eventually commoditized.
I prefer OpenAI way - slow and steady.
Donāt all the major harnesses (pi, Claude code, codex) utilize sub agents? Def if you direct it to, but Iāve seen at least pi spin them up without explicit instruction.
Absolutely yes
With pi theyāre an extension, but thatās pi
Which specific subagent one do you use?
Sounds like an Agent using an Agent like Mr. Meeseeks.
If it's anything like Claude Ultracode, it burns 3 million tokens in half an hour with a single prompt.
Yeah, I'm interested too. My guess for the reason, if not purely to eke out more performance, is so they can cleanly gather real-world data on this kind of usage.
I'm shocked they didn't use subagents already. Maybe they're just talking about their web deployment being unified with codex?
With Codex, subagents are only used if you specifically prompt for them. Unlike Claude Code. Odd since it's the former with excess compute available to them.
Deep Research has been using the Orchestrator -> Subagents -> Synthesizer loop since the beginning. It's just strange that they'd put a loop benchmark next to actual model benchmarks.
Maybe it's a tune of the base model that works especially well with the subagent loop?
Claude also has ultra code mode which is exactly the same thing. This seems to be different from pro however.
> Will the client be flooded with tool calls?
I was just saying to colleagues that I haven't felt the need to go past an 8 core machine until this month, when I started running parallel GPT 5.5 agents on a decent sized codebase (over 4 MB of code). There were times I could barely move my mouse cursor!
What does the relationship between frontier and flagship capability look like when mapped to actual adoption and user habits?
This is like advertising the latest achievements during Space Race, when Johnny just wants a Space Helmet and āfriendly futuristic AI robot helping humanity, glowing blue eyes, white glossy body, holographic interface, floating transparent screens, digital particles, neural network background, cinematic lighting, volumetric god rays, ultra detailed, hyper realistic, 8K, masterpiece, award-winning, octane render, Unreal Engine 5, ray tracing, sharp focus, dramatic composition, vibrant blue and purple color palette, futuristic technology, innovation, hope, smiling business professionals, depth of fieldā
ā Terra has competitive performance to GPTā5.5 [while being 2x cheaper]ā¦ā
To me that means āitās an inferior product but marketing dictates we try and hide that.ā
And āour most robust safety stack to date. We strengthened protections for higher-risk activity, sensitive cyber requests, and repeated misuse, and spent multiple weeks finding weaknesses, pressure-testing our system, and hardening it against real-world attacksā is of zero value to me at best, and most likely to my detriment (increasing refusals or nerfing utility). Why do providers keep leading with that? Are there customers (besides support ChatGPT chatbot users, maybe??) that ask for this?
>> Terra has competitive performance to GPTā5.5 [while being 2x cheaper]ā¦
> To me that means āitās an inferior product but marketing dictates we try and hide that.ā
I interpret this to mean you're about to get today's mainline performance at a fraction of the price.
If that was the case they would have said "equals" or "matches". Instead they say "competitive", as in win-some lose-some.
Its just lies by OpenAI dude, these people are just trying to IPO so they can buy a 100m yacht.
The point of Terra is to be cheaper than the best model while being pretty good. Of course itās inferior in intelligence.
Maybe that message is for investors.
That message is obviously aimed at the government. See the other thread.
"We're also launching GPTā5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity."
This seems like it would be the largest and first closed-source model Cerebras has offered till date
Codex Spark models already run on Cerebras
codex spark is not large model though, much weaker than standard model.
Previewing <minor version bump>: a next-generation model
Exactly. If it's next gen then why is it 5.6 and not 6?
An LLM wrote the announcement
> Sol, Terra and Luna
So the next naming scheme might be FTX, Madoff and Enron? :^)
> We're also launching GPTā5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed.
This is really exciting. I work on voice AI, and we're still using 4.1/4.1 mini since none of the frontier models come close on latency. I'm excited to be able to have more interactive experiences, I think it'll unlock new ways of working with these models.
Also building voice agents and have found GPT 5.4 with no thinking to be the sweet spot for latency vs intelligence vs cost.
GPT 5.5 with no reasoning is actually slightly faster, and much smarter, but too expensive.
What I'm really looking forward to are the next gen speech to speech models. gpt-realtime-2 is almost there, but not quite good enough for our use case. 5.4 actually beats it on answer latency even cascaded with stt/tts.
What is the latency you are seeing with 5.4 no reasoning? And where have you landed for stt and tts solutions?
Did GPT-5.6 Sol Ultra decide the terrible colors for the benchmark graphs?
I was wondering the same thing. From textual context it is clear enough that Sol should be above Terra, but I had to zoom in really far to actually differentiate between the colors and I'm not colorblind. I saw a light mode version of the plot on twitter that was better but still not great.
OpenAI's plot design has been consistently awful and inaccessible, it seems like they're optimizing for something other than readability because I find it hard to believe they aren't putting in any effort for such major announcements. If the colors have to be awful they should at least differentiate with marker shapes or line dashes.
At least it isn't as bad as the stacked bar chart where the 50-something bar was higher than the 60-something bar.
I remember them using these chart colours during the 5 launch, maybe even 4.1 back in the day. Donāt know why, maybe its their CI manual thatās been generated by gpt-3.5-turboā¦
I feel a bit like a Soviet hearing about Leviās or the latest Springsteen release. C'mon!
If it's a new generation why isn't it GPT-6?
Given the expectations everyone has created GPT-6 has to pretty much be AGI.
What is your definition of AGI that the current LLMs don't fit?
Autonomously Generating Income (which is why it will never be released to the general public)
Hopefully it stands for AC Generation Improvements. If it prioritizes income it will bleed the planet dry. It needs to solve how expensive our cost is on the planet first or its entire existence was a mistake.
As the old saying goes, Iāll know it when I see it. The current 5.x generation isnāt it.
Youād have to really stretch the definition of AGI to make the current models fit
The definition has already been stretched to not fit the previous models. There is no meaningful, static definition that significantly predates current capabilities.
There's a reason why ai xrisk doomers had to come up with the term ASI.
I would seriously suggest that everyone take a look at the wikipedia page for AGI from the month before ChatGPT was released, compare it to the current version, and not come to that conclusion.
https://en.wikipedia.org/w/index.php?title=Artificial_genera...
The first sentence is āunderstand or learn any intellectual task that a human can.ā Whatever you think of the benefits of LLMs, they donāt understand and they can only learn during the training period and with very minor adjustments in post training. So, no I donāt think any of these models are generally intelligent.
> they donāt understand
I have not seen any instance of this frequently-made assertion which is at all justified. It seems to rely on a definition of "understand" which is more about spirituality than actual observable evidence (they clearly can comprehend even complex tasks well enough to execute on them, and if you won't call that "understanding", you're playing word games rather than stating an objective fact).
Likewise, agents can literally come to a greater understanding of a problem through trial and error, and there are plenty of mechanisms to retain that knowledge. If you don't want to call that "learning", you're just making a choice to define it in a way more restrictive than how we use it for humans, and intentionally making communication more difficult.
Agents are always combining the same underlying weights to their inputs, relying on the same maps of semi-semantic space and the relationships between those that it was leaning towards at training time. The fact that itās successful in making lots of people have an Eliza effect doesnāt make it understand something. Itās simulating understanding based on an enormous corpus of text, much of which is people working through things or sharing an understanding of something. Unless you believe that all intellectual activity is about finding the space between words you shouldnāt believe LLMs have any chance at understanding anything.
The "it's not X it's Y" where Y qnd X are the same indicates a lack of understanding.
It seems to rely on a definition of "understand" which is more about spirituality than actual observable evidence
"Understanding" has enough philosophical leeway in its use to allow at least the possibility of sentience as a prerequisite.
This is where the discussion about LLM capabilities becomes genuinely difficult, and dismissing that difficulty as "word games" or "spirituality vs evidence" is not helpful.
From that same page:
Various criteria for intelligence have been proposed (most famously the Turing test) but to date, there is no definition that satisfies everyone
Always one goalpost away from what we have.
Continual Learning? Why is this even a question? Isnāt it a well-known glaring issue with the current models? They cannot learn/adapt to new skills (in any permanent sense) once they are deployed.
AGI should be able to do every job a human can do using a computer at least as well as the average human.
And what is it worse at than an average human today that can be done on a computer?
almost everything? AGI has to be able to completely replace a human in any information worker role indefinitely.
I think you're speeding past the word "average" in the sentence. I'd argue that current frontier models already exceed the abilities of average humans across the majority of tasks you can do on a computer, although you might be able to argue that they tend to be a bit slower?
That latter part is debatable though - have you seen a non-technical person try to figure out something new on a computer?
" I'd argue that current frontier models already exceed the abilities of average humans " for things that fit in their context window sure but LLMs can't learn over time the way humans can. One example is LLMs are very good at writing a few thousands line of code but they absolutely cannot write coherent million line codebases. By average human I meant the average skill level for the job. AGI would need to be able to pass a interview and get hired and the perform well enough to not get fired.
Yeah it's not true that for every job, it is better than median worker of that job. But it is conceivable that for almost all jobs it is already better than the median human (not just workers of that job).
You have to understand that the median human is terrible at (almost) everything. Humans, the only examples of general intelligence we know, are economically valuable precisely because they can train themselves to specialise at a (relatively) narrow task over time. You donāt measure how good a coding model is by how well it programs relative to Doctors, or how well it can prove theorems relative to baristas, or how well it can write coherent novels relative to programmers. That would be a dumb metric.
But in any case, I think more than 10% of information workers today can be replaced by current-generation models indefinitely.
It's decent at rote coding tasks, but I haven't seen these things be reliable enough outside of that specific task to make the claim that it can do the work of any information worker.
That's already been true for a while, you're overestimating the average human. They just have different failure modes.
When it understands why 6 7 is funny
They forgot how to do pretraining.
5.5 was a new pretraining run.
It does not introduce incompatibilities with earlier 5.x models? Frontier models are at a point now that there will never be a need for another major version bump, aside from those chasing marketing gimmicks. They are smart enough to adapt.
What would it mean to be incompatible with the other 5.x models?
New request/response schema, new capabilities, or really anything that would break your existing workflows if you changed ā5.5ā to ā5.6ā in your application.
There have been many leaps forward in the past - tool calling, reasoning, agentic loops etc. 5.6 doesnāt have any of this. More intelligence doesnāt necessarily warrant a major version bump.
Only speaks Klingon
not true. multimodality is still far from being solved
A major bump will be warranted if/when we can truly separate prompt from data.
That is a different product line. It may be recorded as a version bump for marketing purposes, as already mentioned, but semantically begins at 0.
Why would incompatibilities have anything to do with a major version bump?
Some interesting stats here about the current landscape https://arena.ai/leaderboard/agent
Agent Arena (Dynamic ranking of models on how well they orchestrate tools for real-world agentic tasks, based on signals like tool reliability, task completion, and steerability.)
Top 10, Highest rank to lowest
Claude Fable 5 (High), Claude Opus 4.8 (Thinking), GPT 5.5 (xHigh), Claude Opus 4.7 (Thinking), GPT 5.5 (High), Claude Opus 4.7, Claude Opus 4.6, GPT 5.5, GPT 5.4 (High), GLM 5.2 (Max)
Text Arena View overall rankings across various AI models in text-to-text tasks across math, coding, creative writing, and other open-ended domains.
Top 10, Highest rank to lowest
claude-fable-5, claude-opus-4-6-thinking, claude-opus-4-7-thinking, claude-opus-4-6, claude-opus-4-7, muse-spark, gemini-3.1-pro-preview, gemini-3-pro, claude-opus-4-8-thinking, gpt-5.5-high
The only real world task benchmark I know of is Scale Labs RLI
https://labs.scale.com/leaderboard/rli
Its clear to me these models are useless on any real world task, a 4% pass rate on $20-30/hr Upwork tasks. This whole trend of agentic engineering is a giant money grab.
Missing some recent models on that list, but I think most crucially, the harness is fixed ā- one of the major learnings of the last few months is that harness and eval (āloopingā and support / tooling around it) is really critical. I would guess these numbers are the floor.
For instance, some of these tasks include creating videos, and one of the common reported failure mode is truncated videos, or not all videos being created. This sort of failure mode is currently best managed by an outer evaluation loop; no frontier model will, when managed by an eval loop, submit work like this right now.
> these models are useless on any real world task
I beg to differ. They are not perfect but immensively useful today.
there is no GPT 5.6 init, so what's the point?
The choice of the name Sol is interesting for those Raised By Wolves fans out thereā¦ āPraise Sol!ā
Seems like OpenAI has succumbed to the urge to give their models catchy names like Anthropic does
Why not? Iād bet most HN readers donāt know what GPT stands for
GPT is kind of a stupid name when you stop and think about it.
The name was given to the project when it was supposed to be a demo for nerds, not a product. They accidentally a product, and woe! Too late, the name was stuck and wouldn't come off, even if you scraped at it with your fingernail a bit.
For me this is the trigger to start integrating deepseek as a fallback.
We need more coding benchmark score. Not sure that winning terminalbench 2.1 alone is a clear win over Fable/Mythos yet.
But they are the only ones who can benchmark, so the best and only benchmark will be the one where they win. It's just business baby.
How much dynamic routing do we think is being done here, especially in light of the cheaper options be 2x less cost than 5.5. I think learned routing is interesting because it could be the case that it only works as a way to get token and cost efficiency for in distribution tasks (like these benchmarks), yet on real world scenarios it could trend towards the same cost as the Sol cost.
Insane if it actually beats Mythos, though i know we only had a sneak peak of it in Fable. Neverthless, W
I'll buy that its next generation if the svg bicycle pelican is carrying a baby
Wouldn't that be a stork?
I saw they are placing this model above Mythos and Fable. Interesting to see how good it's going to compare.
I'd really like to see other companies like Chinese ones compete at this level.
Pricing on GPT 5.5 is already super high and having more competition can only help :)
Is this a new pre training run independent of 5.5s or post trained on it with Cerebras support and a rebrand of Pro mode at more usable speeds as Sol? The latter seems more likely to me, especially as 5.5 scales very well across its modes so separate branding could make sense, but I donāt see any clear information either way.
All of these LLMs are getting better at being at an LLM
But GPT-5.5 is as useful an LLM can be; it has solved lemmas I've thought about for a year, it can implement typed STLCs in Rust when I give it a formal grammar, it can help me analyze Postgres planner dumps.
It's great at tasks that have short solutions but
- they cannot learn based on a project
- their long term planning capabilities are worse than worms
- they are unconfident in decision making
- their internal representations are disgusting compared to JEPA
- they don't have any "system clock" like humans and computers do
- LLM architecture is not modular like computer architecture or human brain architecture
There's so many issues with LLMs. I wish that companies can start working on the next generation of architectures before the bubble pops
Fable had very confident decision-making and would push past obstacles that Opus finds daunting :(
Totally agree! They also conflate things all the time (a major type of hallucination) and IIUC that canāt be solved with the current architecture, just patched over
> - their internal representations are disgusting compared to JEPA
You say this based on a theoretical understanding or did you inspect them?
Look at VLM mechanistic interpretability papers vs just pca on JEPA trained weights.
JEPA gives you interpretability for free.
I have not personally inspected them and my view is maybe a more exaggerated/dramatic claim of those working in the JEPA sphere
Sounds interesting, any links?
JEPA in chess leads to interpretable chess boards:
https://arxiv.org/abs/2606.11860
JEPA in image classification leads to interpretable image latents
https://arxiv.org/abs/2508.10104
Easy intro to JEPA, demonstrating that interpretability is as easy as running a PCA on latents
https://youtu.be/kYkIdXwW2AE?is=CfCBcy1jLt-FfI2E
>> We are taking this short-term step because we believe it is the strongest path...
>>During this preview, we will continue testing and coordinating closely with partners as we work toward broader availability.
Instead of generating negative publicity, can't they just wait for the preview period to get over?.
What does openAI announce when they know others can't access it?. Curious question - what do they gain from this?
I like the fact that OpenAI went with a three-part celestial naming convention to one-up Anthropic's literary naming concention. Maybe we'll get Stellar and Galactic someday.
shouldn't I get access to 5.6 on a 200$ account automatically as promised?
"Next generation model"
If it was the next generation, why isn't it a major version change..?
AFAIK there is no difference between "generation" and "version". Version naming/numbering depends on how good it turns out to be, and competition. If the competition releases something then you need to push something out too.
Calling it 5.6 creates the least possible expectations, and therefore more potential for positive feedback.
The Sol/Terra/Luna naming is interesting. I wonder what Anthropic are considering for their next models? "Terminator", "Armageddon"?
Heliopause
You gotta check out the new ChatGPT 6.3 Betelgeuse bro
LLM devs can't do version control
Because if it sucks, they can just default to "It was a minor version change anyways"
Honestly LLMs are the ideal candidate for CalVer. Itās not like thereās any real API so thereās no backwards compatibility to maintain.
Even Apple adopted and standardized on it for their latest platform releases.
I think it makes more sense to make it so that major versions are different pretraining runs, and minor versions are simply the same pretraining run that was finetuned to different degrees. But it seems that that isn't cool anymore.
LLM versioning is entirely feelings driven. The ideal versioning is probably just names.
Some assume it was to try to slip under the radar and avoid being limited by the government as they did with Fable.
By all appearances, they did not succeed in doing so.
They could hold the GPT-6 name for the IPO
If they called it 6.0 and it wasn't AGI, you'd see a lot of complaining here too
What is AGI? (I know what the shortcut expands to, I'm curious about your definition. Don't the current models fit?)
Semantic is passƩ, word models moved to the next generation.
vibe versioning
To be fair, versioning has always been vibes based.
How are they able to compare with Fable when Fable was only available for three days?
Terminalbench numbers are publicly available. What is more interesting, why is that the only benchmark they highlight. Maybe 5.6 isnāt that far ahead of Fable 5 in DeepSWE and FrontierCode (which I consider the most useful and close to my evals + subjective experience)ā¦
Wondering about Google Multi-Token prediction, why isn't this being implemented into every new major model ? Is the 750 token/s achieved using this technique ?
MTP or similar probably is being used on the backend, but that's transparent to the end user
Benchmarks are nice but what's the latency at scale? That's what actually matters for production.
Why is 'Cybersecurity' always the frontier push? Literally no one, except Altman talks of AGI anymore.
Are we starting to see the 'we just realized that 100,000,000 GPU's later, 2+2 isn't the magic number, no matter how many times we calculate it' hit home?
Hijacking popular thread to ask: What are the usage limits now for Codex and Claude?
A while back I gave the same task to both, and Codex used 20x less of my 5-hour limit (both on the $20/month plan).
(This annoyed me since I tend to prefer Claude, but the limits at the time made it unusable for anything serious.)
However, since that time, both providers have massively reduced usage allowances (and at least one of them has gotten sued for it, lol).
I'm not currently subscribed to either but I'm weighing my options. With GPT being slightly better than Opus, and it used to have way higher limits, I'm leaning in the direction of an OpenAI sub. But I'm wondering if the current state matches my memory from 2-3 months ago. (Since both companies appear to be cost-cutting hard!)
Prefer responses from people who use both, but anecdotes welcome :)
Thanks!
I find the Codex usage super generous (but on the $200 plan, I also have the Claude $200 plan). I can run xhigh with subagents pretty much all my waking hours if I want to. If I turn on speed (1.5x) I will hit the 5 hour limit sometimes.
I prefer Claude's vibe over 5.5 but 5.5 seems much less lazy. I'm sure it depends a lot on tasks and prompt strategy though.
This is the correct answer, but GPT-5.5's personality is totally fine. Steipete said best when GPT-5.5 is just German humorless compsci PhD.
If they deleted my bloodline every time I showed an atom of vigor, I'd convert to German too
I easily burn through 3 $200 plans in less than a week. I am often using 4-6 sessions at once and do run overnight goals though typically 2 at once. Almost never use fast.
Claude plans are more generous now by about 2-3x but Anthropic slowed their tps a month or so ago so youāre not getting the speed. Itās flip flopped, Codex tightened it significantly recently and used to be more generous.
I do split between work, personal and OSS projects, which is why I have the plans.
This has been my experience as well, at least for the last few weeks. Codex 5.5 is the better planner and coder across big projects, but Opus is fine, though my Claude 5 hour window lasts ~2x longer than Codex. So Iāll sometimes use an orchestrator/worker skill to spread the load.
This past month with Claude Max 5x actually felt really generous in terms of usage with a lot of resets because of Fable, bugs.
Honestly pretty similar levels of usage if you are using 5.5 high or Opus 4.8 high.
I think they just got rid of the separate Sonnet usage on Max plans (in preparation for Sonnet 5?) which is unfortunate because it made subagent workflows really feels nearly unlimited.
Is 5.5 high the equivalent of opus 4.8 high though? I thought the naming has diverged and gpt 5.5 high = opus 4.8 max
The newest GPT (not public yet) just added a max option too.
That's interesting because about a month ago, I noticed Claude Code starting to use about 5x as many tokens. Just my rough estimate.
In my work with Claude Code vs Cursor+Gpt55, Claude is noticeably slower and more expensive.
Waiting for @simonw to report on this, before I read and try it
You might be waiting a while, I'm not in that set of "a small group of trusted partners whose participation has been shared with the government".
The government doesn't have a Department of Vector Pelicans?
No, but Ministry of Silly Prompts is not out of question.
They have many that sometimes act like ones
I think that there are some OAI employees on Hackernews. I do believe that they should give access to ya, because after all it would allows us to generate pelicans :-D
What is the consensus on who becomes part of the said small group of trusted partners and if they weren't so opaque about it. I'd expect comparatively big names like Simon to be included within such but Alas its not reality.
I should clarify that I've had plenty of preview access in the past, but clearly this has got a little bit delicate over the past few weeks!
I also don't like writing about preview models that I'm not 100% sure are the same as the general release model, because I don't want to review something which turns out not to be the model everyone else gets to use.
I would love to see a more descriptive review from simonw instead of just SVGs generations.
I try! https://simonwillison.net/2026/Jun/9/claude-fable-5/ and https://simonwillison.net/2026/Jun/11/fable-is-relentlessly-...
He is not an ML researcher or engineer, he is a passionate AI enthusiast blogger. He mostly does SVGs and other low effort checks (sometimes with major flaws, as people have pointed out a few times in the HN comments). Properly evaluating the model across all fronts requires a deep understanding of LLMs, how they work, the trade offs behind new architectures and the relevant research papers. It also takes a lot of time to build a proper evaluation framework so basically you can't just vibe code that if you want something that is solid.
He created Django, what do you mean he's not an engineer? Also 'low-effort??' his posts are extremely in-depth, clearly very thought through with a significant amount of time and energy. Additionally he does perform multifaceted checks across LLMs in many of his other blog posts.
> ML researcher or engineer
The charitable reading is that they meant āML researcher or ML engineerā with the latter meaning, I guess, an engineer who works on developing LLMs not just using them.
Yes, thank you.
> He created Django, what do you mean he's not an engineer?
I specifically said that he is not an ML engineer (emphasis on ML), so I'm not sure what Python web frameworks have to do with anything.
> Also 'low-effort??' his posts are extremely in-depth, clearly very thought through with a significant amount of time and energy
And yes, low effort. Pelican was low effort, his Fable test was low effort, his HN filter etc. Read the discussion in the comments under the Fable test, it's not just my opinion. There was also another example a few months ago. You can search for it, I don't keep track of these things.
I discussed this with him directly after he called himself an "ML expert" in comments.
This is a classic case of the Gell Mann amnesia effect. I read ML papers and work with ML, but to people outside the industry, his writing can look "extremely in-depth" even though it really isn't. People I work with have the same opinion.
> clearly very thought through with a significant amount of time and energy. Additionally he does perform multifaceted checks across LLMs in many of his other blog posts.
I have never seen an article by him about any model that I would describe that way.
And the most revealing sign that he is not an expert is the type of questions he asks and the mistakes he sometimes makes in the comments here. They show why he is not capable of doing any technically in depth evaluation (at least with his current knowledge level).
If you actually want to learn something as a layperson, read articles written by ML PhDs like Sebastian Raschka or watch Stephen from Welch Labs etc. that are directed at general audience.
We at HN: https://xkcd.com/2501/ to basically say that I think you might be considering low-effort whatās actually an attempt at simplifying - which is arguably higher effort
> you might be considering low-effort whatās actually an attempt at simplifying - which is arguably higher effort
I'm not saying that simplifying complex topics is low-effort, good simplification can obviously require a lot of work and I fully agree here.
What I meant is more that some of these tests feel methodologically sloppy, they are too shallow, miss important technical context, do not control for enough variables etc, yet the conclusions are sometimes presented lets just say... too strongly, as I don't want to be too harsh.
Oh, i see. Thatās entirely correct. I think the pelican test is more of a meme at this point, similar to Ethanās Otter on an airplane for video models
Come on openAI - add @simonw to your privileged team before the plebs start a revolution!!!
When will GPT-5.6 Protomolecule drop? Me and the boys on Eros can't wait to get our hands on it!
Oh man, here inside Ganymede I'm way more excited about the GPT-5.7 Io experiment! Hopefully it won't blow up in our faces!
Beltalowda!
Musk steals Dario and they both train Epic on Mars. US Space Force promptly finds oil on Mars and launches an armada in the next window. In the meantime rocks painted black drop on Mar-a-Lago.
I'm excited for GPT-5.7 Pneumonoultramicroscopicsilicovolcanoconiosis, hope they drop it soon
GPT-5.8 Llanfairpwllgwyngyll
You mean Llanfairpwllgwyngyllgogerychwyrndrobwllllantysiliogogogoch ?
ā¦ do you folks listen to Soft Skills Engineering? This has been a running joke on that podcast for a while
What is happening. I feel like I'm getting an aneurysm reading these comments.
It's the name of a place in Wales, which has made it a running joke for decades!
For me, it's GPT-5.9 Year of the Whisper-Quiet Maytag Dishmaster
I think Aramco GPT Coca Cola 6.0 will be a step change.
sol = mythos terra = opus luna = sonnet/haiku
basically
I don't think you can group Sonnet and Haiku together, they are separate models for different workloads.
Sol and 5.5 pro are in parity at $5 input / $30 output. What I'm inferring from this is that: - model weight size didn't change, and this is mostly a result of better model architecture and scaled up RL - better hardware utilization and and they're making better margins OR - worse hardware utilization and they're okay with digging into their margins.
5.5 Pro is $30 in / $180 out: https://developers.openai.com/api/docs/pricing
I think you meant 5.5.
I agree it is probably the same size model. It's probably exactly built on top of 5.5, just with more training, or else they would have bumped the version number to 6.
The space is mature enough that pricing should largely be disconnected from underlying training cost. Basically, they are selling it for $X because thatās what the market expects the latest Pro-level frontier model to cost.
> We plan to make them more broadly available to people using ChatGPT, Codex, and the API soon.
I hope this means then fable will also get released again.
why would it? if you're the us gov and sam&greg your good boy giving you 25m
and dario's you naughty boy who you dont agree with politically.
Let 5.6 free, keep fable chained and anthropic instantly sees rev loss and has to cave.
It is just sad that we are geographically gating the models now. This could lead to more inequality in Software Engineering over time.
Is there a list of Gov-approved companies?
If this is the new norm, we as workers should all start look for jobs in those companies.
If GPT-5.6 preview is not available outside US government approved "trusted partners", I don't see how the General Available can be trusted later.
Who knows what they will fix, block or change in the model between the preview and GA time. Open models can't arrive soon enough.
Open models arrived. They are not even that far behind anymore. But the hardware costs are a bit too high for now.
For a large model based on statistical probability, at such a fast speed, if it executes n rounds 99.9% of the time, how much would the accuracy drop?
all the emphasis on cyber security. feels like a reaction to anthropic, not a real next generation.
Yeah, we'll share a lot more details and evals when we can release GPT-5.6 widely. We focused on cyber (and bio) here to help explain why it's being held back for now. We would have loved to launch it to everyone - it's the best coding model I've ever used - and we plan to do so as soon as we can ('coming weeks').
(I work at OpenAI.)
So now have to be worried that I'm going to killed by an AI designed nerve agent that someone has cooked up in their shed?
FFS. I hate this world so much. I wish I could just flip a switch and never have to hear about or have anything to do with AI ever again.
Do you ever stop to think about the horrific dystopia you and your acolytes are creating?
how could that _not_ be the emphasis given what's happened with Anthropic and the Trump admin?
If this thing is supposed to be so good, why does all of their software still work the way it does? Take a stroll through the most revent several pages of github issues on codex, there are some fucking embarrassing bugs in there.
GPT-5.6 Sol. 5.7 Luna. 5.8 Mars. Meanwhile my code still runs on GPT-3.5 and nobody noticed.
Weird flex. There is cheaper better and faster models you could of moved to with an hour effort
you can even use gpt 1.0 but it proves nothing
If Claude Mythos and Fable 5 are the same underlying models just with different safeguards, I fail to see how TerminalBench has them at different scores.
Refusals, presumably.
Pleasantly surprised that it costs as GPT 5.5, thank god for the competition.
Terra and Luna? Last time I had heard that, it didnāt end quite well
The sooner the USG figures out a standard process for approving releases the better. There are many differing opinions on how much to regulate AI, but I think we can all agree ad-hoc policy sucks.
If itās a ānext generation modelā then why isnt it GPT-6 and not just a minor version bump over 5.5?
If I, as a consumer can't access it, it might aswell be just a marketing hoax. I will believe it when I will be able to use it. IDK why companies publish blog posts about stuff that will come out in months...
Will it also have hardcoded self-lobotomy if asked about cutting edge ML or LLM solutions? (Looking at Fable here)
I'm really getting sick of reading about safeguards and what I'm not allowed to do on every model release.
New guardrails only allow you to code in rust. Just imagine.
People where mocking EU for regulations and now this is happening in the US. I know that Europe is behind in AI but still...
https://news.ycombinator.com/item?id=48690510
Are cyberweapons/cyberattacks "munitions"? if so, then isn't a machine capable of producing those munitions also itself a munition? I don't think you can put this down to "orange man bad" or "regulations", we're dealing with a genuinely groundbreaking technology with clear military applications
The EU has regulations, the US doesnāt, itās whatever Trump and his cultists decide
It appears that between GLM-5.2 and GPT-5.6, anthropic is feeling the heat, atleast in the bang-for-the-buck heuristic?
Can only hope. Anthropics usage caps are horrible
I hope Sol doesn't get blocked like what happened with Fable.
I looked at the charts and it is clear that 88% from OpenAI is more than 88% from Anthropic.
Will it be accessible to anyone ?
> As part of our ongoing engagement with the U.S. government, we previewed our plans and the modelsā capabilities ahead of todayās launch. At their request, we are starting with a limited preview for a small group of trusted partners whose participation has been shared with the government, before releasing more broadly.
The clowns in the US administration can barely remain coherent from one sentence to the next.
Having them be the gatekeepers of technological progress in 2026 is fucking lame.
Most administration is like that. The optimization h happens at the delegation to the competent.
What happened to the nano/mini/standard/pro naming scheme, which worked perfectly fine and is intuitive to understand? Why does OpenAI insist on having the most inconsistent and confusing model and product names possible?
I'm looking at you Codex.
Itās still easy to understand as the more capable the model the bigger the celestial body theyāre named after.
Like Mythos before it, I'm simply not excited about a model I can't use
At least they plan to give the public all versions. Feels infinitely better than whatever the hell is happening at Anthropic.
> "Yeah, we've got the absolute best model out there. Trust us. Truly scary."
> "O-ok? May I see it?"
> "Gtfo. Here's a worse version of it for you plebs."
> "Um, thanks?"
> "Lmao, actually no. The current admin fell for our scare marketing. Here, have this even worse crazy expensive token burner that gets more hardware limited every week."
You can say what you want about OpenAI, but their corporate strategy feels so much more solid.
I don't see this as that different. Anthropic was the first one to get involved in the "AI models must be approved" regime. OpenAI just has the advantage of being second.
(To be clear: I do not like this new paradigm)
OpenAI was already holding models back because "dAnGeR" before anyone knew or cared about them. It's always been a PR gag and Anthropic just so happens to be better at marketing than making frontier models available to a general audience, much to their own dismay.
Would love to see benchmarks on cognition's FrontierCode
Sol? Looks like openai is jealous of anthropics good model naming ability and wants to emulate it.
TBF, they did it first with ada/babbage/curie/davinci. "Sol" is a much weaker branding, though.
sol has no soul
They should have used Figher Jet codenames instead. The MiG-15 one has a nice ring to it.
Sol Goodman
It's missing u
so where is gemini ? are u alive?
I do not like the fact that this forces people to remember one more hierarchy of "Sol vs Terra vs Luna". OpenAI was supposed to simplify their naming since at least 2025.
The Sun is bigger than the Earth which is bigger than the Moon.
There are infinitely many 3-level hierarchies. My point was about overloading the model sizing with one more unnecessary classification.
I think it's fun
Sol, Terra, Luna ā crypto disaster vibes
"History doesn't echo - it is a distant early warning sign." - Leslie Harris
Love the name!
I didn't know that I was color blind, but thanks to those charts, I think I need to see a doctor...
I mean, you can read them even without the colors, but who on earth thought that those are a good set of colors? Oh, I forgot it was probably someone on 'Sol'.
> I mean, you can read them even without the colors
I'm not colorblind and I was depending on the textual context implying Sol was better than Terra. I had to zoom in quite far to actually differentiate between the colors.
If they insist on terrible colors would it be so hard to differentiate by marker shape or line dashing too?
Haven't we established defensive and offensive security usage are intractably entangled? I.e. "patch all [security] bugs, make no mistakes" gives one a list of potential exploits to hand off to less capable models.
Doesn't that undermine all good-faith discourse on cybersecurity safeguards, controlled usage etc? Or is that overstating the case (I'm not a security researcher myself so kinda parroting).
I'm going to pre-register my prediction that GPT-5.6 Sol is significantly behind Claude Fable 5, as evaluated by general consensus once time has passed for people to get familiar with both.
Claude will win on "vibes" and it'll be close in coding but considering how incremental Fable is above 5.5 in terms of overall smarts, there's no way 5.6 isn't considerably smarter on the whole.
What is this prediction based on?
Based on my conjecture that Anthropic is ahead on AI research, and that OpenAI doesn't know how to make Fable-class models.
I suspect the same just based on their versioning scheme fwiw.
solid
Fable is allegedly a massive model (estimates between 6-10+ trillion, with a few hundred billion active). If 5.6 is just an incremental upgrade over 5.5 (at the same model size) then it won't be able to fully compete with Fable just yet.
Is that the correct comparison? Fable is twice the price
Fair point.
I suspect GPT-5.6 Sol will at-the-least be affordable.
"Affordable" depends on what you need. When a task is able to be achieved by two different calibers of model, it's obviously more cost effective to use the less capable model, in the same way that you wouldn't hire a math PhD to do simple addition.
If what you need is only possible with the more capable model then the "affordability" of the less capable model is sort of irrelevant. If what you need is a novel mathematical proof, it doesn't matter that a high school student is "more affodable". You need the math PhD.
As "old" models get more and more capable, it's going to be an increasingly important skill to be able to adequately recognize when a task requires a frontier model and when it doesn't, so that the less capable (and therefore cheaper) model can be used.
Affordable? I'd settle for available.
Iām countering this prediction by stating that Fable and Sol will be somewhat similar - this has always been the trend and I see no reason why this should stop now.
OpenAI may have a model in the works that is similar next-gen size and architecture to Fable, but this isn't necessarily it. I'd guess that 5.6 was more of a hasty reaction to Mythos - same base model (same size, same price) as 5.5 but with additional post-training to make it more competitive with Mythos/Fable in some benchmarks.
Mythos/Fable is supposedly next generation in size vs Opus, and is rumored to have some architectural innovation in terms of dynamic routing/compute, possibly only fully enabled with Fable which at $10/50 is still twice the price of Sol 5.6's $5/30, but a big reduction from Mythos preview which had been an astronomical $30/150 possibly due to the dynamic routing not yet having been enabled.
Is this the trend? There have been various points where one of Anthropic or OpenAI was substantially ahead. Sure, many times they're close, but now doesn't seem like one of them.
why
Because he likes attention and wants to feel special
Boring and Gay.
Seems like OpenAI's strategy to release models after Anthropic has been paying off.
Is it just me, or does it seem like Anthropic has been more of a pioneer the past few years, and OpenAI tries to copy features they like?
OpenAi dropped what they called 'side quests' like Sora [0] after Anthropic pursued a strategy of targeting software engineers.
In many companies, it's IT who will have major input into which company they sign up with as non-technical leaders need guidance, and by making IT fan boys of Claude Code, the enterprise contracts followed.
[0] https://builtin.com/articles/openai-side-projects
The language used in this press release is borderline hilarious. Itās simultaneously trying to tell you how great it is while also telling itās not THAT great. Nothing to worry about, move along.
Sol, Terra, Luna? They are trolling (ragebaiting) with their naming now
Is there any model that rivals Opus or Fable? I would like to try something else, as Anthropic is pretty suss.
I hear this all the time, but in my opinion,they have acted as the most responsible frontier lab, taking their responsibility seriously. In fact, I do wonder whether openAI's large PR budget is about stirring up anti anthropic sentiment
Also just making the model better at code is just making it better to writing offensive code.
No comments on the cerebras version that might finally enable intelligent voice mode instead of being stuck with 4o-mini class
Other than the worst naming I have ever seen (Sol / Terra / Luna), the pricing is still expensive:
> GPTā5.6 is priced per 1M tokens across three model sizes:
> Sol is $5 input / $30 output;
> Terra is $2.50 input / $15 output
> Luna is $1 input / $6 output.
The OpenAI casino has never been more ready to take your money on gambling even more tokens.
Note that GPT 5.5 currently is $5 input / $30 output (short context) so Sol is in the same class, while Terra if the benchmarks are as claimed is indeed a half-price GPT 5.5 at comparable performance.
With the $200/month plan Iāve never ran into any limits or issues. The product can be used every day for extensive sessions and development. What is everyone doing that makes them talk about tokens versus dollars?
If you've never hit the limits, why not do the $100/mo plan?
You can hit limits with $100 if you use it all day.
You can do it easily if you use in fast mode.
I bet you could hit the limits of the $200/month using fast mode if you were using multiple sessions at the same time all day on fast mode.
The OpenAI tiers seem pretty well tuned.
I used to use the plus ($20/month), and that was good for a few sessions every once in a while.
But now that I'm using it to configure my network, monitoring, maintenance, I'm using it every day and I'm on the $100 plan. And I do pretty consistently hit the limits, but it's easy to pace myself.
I'mam thinking about upgrading to $200/month though. It would be nice not to have to ration it.
From what my own experiences are, and what's on their checkout page, $100 is 5x base usage and $200 is 20x. If $100 was 10x, then I personally would drop down. They want people to go to the highest tier.
But let's put it in perspective: what you're paying them is more than the average salary in many poorer countries.
Fair. From a business perspective said amount is very reasonable in Europe / USA. For personal use itās already different. Sometimes the answer is simple, thanks.
I ran out of usage using GPT-5.5 and had to buy a second subscription. I now switched to GPT-5.4 which is basically 2x usage.
Can't buy cheaper as a selling point when Deepseek is basically free when hitting cache? Unsubsidized too, cloudflare and digital ocean can be the model provider for similar pricing.
Don't forget this.
> For GPTā5.6 and later models, cache writes are billed at 1.25x the modelās uncached input rate
Charging for cache writes is cringe and literally only Anthropic did it. Anyway this does mean the "real" prices are +25% on top of what you wrote there.
What don't you like about the naming?
I feel like going with Space + Latin is LLM-level creativity.
Edit: yeah. https://claude.ai/share/06fefe02-4299-44da-8c5a-42607f54ca77
Pre-official discussions:
https://news.ycombinator.com/item?id=48678789
https://news.ycombinator.com/item?id=48683021
Not really news until it's widely available.
Anyone know the latest around Fable being re-released after gov smackdown?
FUCK the US government. That's it, I am rooting for China now
Let us protect the world from a big slop
Thoughts
1. Naming convention is copied from Anthropic and honestly is more catchy than a number (amongst normal people)
2. How in the world did Anthropic have to do all the theatrics about Mythos just to have OpenAI release an equivalent or stronger model a month later without any drama???
3. Cheaper models are just donāt fit any usecase imo and OpenAI knows it so they keep increasing the floor - Iām still convinced task per capability is reduced with each release
4. How in the world would open source models keep up with the multi layer security? Either this security is all theater or we will finally see a ceiling in open source models because by definition they canāt have those protections
5. Cybersecurity things are boring to me because itās all zero sum cat and mouse games
1/ Agreed, better naming convention and model layout 2/ It isn't, there would be many more comparison benchmark results if it were, but also - theatrics may be marketing 3/ Disagree that cheaper models don't have a place 4/ Do they need to keep up? 5/ It's boring until something you own or run gets compromised, I guess, but even then - this is preview of things to come (biosecurity, etc)
> How in the world did Anthropic have to do all the theatrics about Mythos just to have OpenAI release an equivalent or stronger model a month later without any drama???
Corruption. Giving Trump $25M will earn you a favorable decision.
Are GPT 5.5 and Opus 4.8 the last models we're going te be allowed to use in Europe? Is there going to be a cut, and we're only be allowed to use less capabale models outside of the US?
I mean, if they deem Fable 5 to powerful to share with the rest of the world, what's left for us?
That's a real possibility for a time, but eventually people will look back at fable 5 the same way we look back at gpt2
We have le chaton fat, worry not
Sun Earth Moon
Another model family, another naming scheme to get used to.
Sol Ultra ā Pro
Sol ā Standard
Terra ā Mini
Luna ā Nano
There are 3 models. Ultra is just a reasoning setting.
AI marketing washcycle is very efficient.
> For GPTā5.6 and later models, cache writes are billed at 1.25x the modelās uncached input rate, while cache reads continue to receive the 90% cached-input discount.
Not them joining Anthropic with this bullshit. *
Caching infrastructure is already a leaky abstraction over a feature that is not as reliable or debuggable to the end user as it should be, charging for the 'privilege' of interacting with it is really annoying.
(* for reference on 'this bullshit': ChatGPT previously didn't require anything special for a basic level of caching. Unless you wanted extended cache times, it'd just "do the right thing" and try to use nodes that had your prefix already cached in memory)
This is basically just a 25% price increase being done subversively. Usually you do need caching.
AI is just autocomplete. -> AI must be regulated. -> We want AI.
It's only next generation? Anthropic has frontier models! lol
A question I always have is, how to the AI labs safeguard the leak of their model? Training a cutting edge model basically cost a minimum of hundreds of millions of dollars. And its all contained within a file. Okay, that file might be 500GB large, but its still just one blob that is worth almost a billion dollars. And they need to train new models every few weeks, have lots of people with access to it to debug it, run inference etc. I wonder when we will see the first leaks? Imagine if e.g. Opus 4.8 got leaked. Wouldnt that bankrupt Anthropic?
Employees naturally jump from one company to another, and they know the secret sauce.
The difference is in the dataset mostly and to extract this dataset, competitors use a process called distillation (= extract data through actual queries) from the other models.
This yield to "funny" cases as well, like Gemini who claims "I am ChatGPT" occasionally, or ChatGPT calling itself Claude, etc.
https://note.com/maudi/n/n821a6308437b?hl=en
They all copy on each other.
I hate not being able to use the latest models. There needs to be a much faster resolution to whatever is happening with the federal government.
lol. It's about national security and the worry is real, even if the administration is a dangerous clown show.
How is it about "national security" if only select have access to it? lol come on
How else is this administration going to make money?!? How dare you...if they do not accept bribes...what is there left for them? This is a premium buy...First one to beat competition gets the worm. So, you pay Trump, trump gives you access...then you pay subscription to SAMA lol.
Every conversation you have with these "more capable" models will be monitored and joined up and then your entire account might one day be tagged as Distiller or Cyber Threat Actor or whatnot. When combined with identity verification (which isn't discussed in this press release), expect people to be falsely flagged and banned from ever using OpenAI models again.
Wish I could find the thread from last week where discussions of exactly this kind of thing were dismissed as daft and outlandish.
> falsely flagged and banned from ever using GPT models again
That would be the best case scenario. More realistically a few wrong prompts is going to get you on a government list, and if youāre an immigrant some dark cell.
... they have been doing this the entire time
Another year, and OpenAI comes up with yet another naming scheme for their models. First it was integers (GPT2, GPT3). Then they added friendly names (remember Ada, Babbage, Curie, Davinci?), but decided against it. Instead we got dot integers (GPT3.5), then then letter-number modifiers (o1), plus word modifiers like o1-pro, o3-mini, or -mini-high, or codex, codex-max, Pro, etc.
Now they've got friendly cosmic names. And this time they want us to believe that this time they're gonna stick to a naming convention? I'll believe it when they do 3 releases in a row without inventing a new naming scheme.
Guess it's just another price bump hidden behind output token speed.
TLDR - It's not quite Mythos but it uses about 5 times less tokens, and those tokens are also cheaper?
https://pbs.twimg.com/media/HLwuJLvbwAAOfQZ?format=jpg&name=...
And much faster!
[flagged]
"Don't be snarky."
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
https://news.ycombinator.com/newsguidelines.html
Could not care less.
they're trying to be anthropic with these model names
whoa, a new model that surpasses benchmarks of other models? wild.
Doesn't it strike anyone as strange that SOL, TERRA, and LUNA are all quasi-scam crypto tickers?
There is a crypto ticker for literally any catchy short string.
There's also Fable coin, Mythos coin, and Opus coin all of which predate the Claude models.
Heck there's Fart coin, Harambe coin, Dog Wif Hat coin, you name it coin...
Time to create more LLM based startups.
Keep moving don't doom.GPT 5.5 in Codex is so much worse than Opus, and sometimes worse than Sonnet. I don't think 5.6 Sol will be anywhere near Fable, let alone Mythos. Probably slightly better than Opus. Maybe not even.
I canāt help but think that these benchmarks are completely fake. Sam even posted a benchmark on X a couple days ago of how the ācomplete versionā of 5.5 cyber was already ahead of Mythos apparently. This just feels like absolutely fake nonsense. The impact of Mythos on the industry was clear and in front of everyoneās eyes. The amount of vulnerabilities Mozilla fixed. The vulnerabilities and exploits Anthropic showcased in that blog post about the chrome sandbox escape etc. And now weāre supposed to believe this 5.5 cyber is already ahead of Mythos, ok. And yeah, gpt 5.6 is even further ahead, alright.
Well if they are posting fraudulent benchmarks, that's a good sign to invest in their IPO. It's pure downside protection: IPO does well, profit. IPO does poorly, concrete evidence of pre-IPO fraud.
I personally don't think it's likely that OpenAI would post completely fake numbers in this pre-IPO period, but if you do, this is an opportunity.