There's a whole mix there.
Kinda?
sure as hell they are lol.
Indeed.
No? No. TSMC is the only foundry with good analog.
CCDs not many, other things yeah.
There's a whole mix there.I wonder what the packaging technology will be for Venice.
Kinda?Very likely, different from client, which means sharing of CCDs will no longer be taking place.
sure as hell they are lol.MLID said that on future roadmaps, AMD is not differentiating between classic and dense cores for servers.
Indeed.From the press release (and the tweet), it mentions the chip was "brought up" which I imply being back at AMD and powered on.
No? No. TSMC is the only foundry with good analog.SF 4nm IOd seems likely.
CCDs not many, other things yeah.I wonder how many IOd and CCd variants there are planned then.
It's targeting Q3 2026, I believe.From the press release (and the tweet), it mentions the chip was "brought up" which I imply being back at AMD and powered on.
So, perhaps, 1 year of debugging, validation and hopefully H1 or mid 2026 launch.
Indeed it does! Probably August, they love August server launches.It's targeting Q3 2026, I believe.
Biggest customers will have supply in Q2 as is tradition.It's targeting Q3 2026, I believe.
Its really the other way around, millions of people are using GPT and other commercially/closed models. Where as most LLM questions/and even lots of image stuff will run just fine on a 16gb GPU from 2020 using distilled models. Meanwhile look at apple AI, they critically depend on GPT simply because they don't have any models or public facing tech of their own.OEMs are pushing AI because Wall Street orgasms at the mere thought of it. Doesn't mean the OEM's actual customers are interested.
Its really the other way around, millions of people are using GPT and other commercially/closed models.
None of that has anything remotely resembling a viable business model.Its really the other way around, millions of people are using GPT and other commercially/closed models
NPU is not magical, it will steal MemBW from the system. So your concurrent use cases example is not so rosy. And AMD in their lemonade server is sticking to using NPU on strix point for prompt processing and gpu for token generation citing power efficiencyNPU into a SOC, save tons of power, use the same ram chips already on the mobo, and still get good enough performance.
Hmm, that should be already doable, just use a batched mode, batch two inputs together, it will actually improve the compute to memBW needs. Unless you want to use 2 different models.I for one really enjoy using different system prompts on the same model to get more varied results. ie, one programmer bot thinks its a webdev, the other I tell its john carmack, now give me code for X. Would be super nice to get both results at the same time for comparison.
Don't companies like adobe and apple make a non-insignificant amount of their money on selling new software? Its really not just people on social media using AI, there's even larger streamers that have an LLM+TTS generator respond to random chatter comments and everyone loves it. Content aware fill has been a thing since like 2015 and people in commercial media still use it all the time.To make memes. Which is fun but silly that it's driving the entire financial system right now.
Takes the same amount of time, you only get so many tokens/iterations per second regardless of how you organize the workload. Idk unless like you're running a bunch of tiny models on a huge accelerator but that's not what I'm talking about. Its much better to have models be as big as possible and thats why you don't see anyone commercially trying to run/sell swarms of 4gb distills.Hmm, that should be already doable, just use a batched mode, batch two inputs together, it will actually improve the compute to memBW needs. Unless you want to use 2 different models.
There are instances where its better when the entire system can fully share resources instead of one component dependent on another. The main bottleneck really is the acceleration of the FP instructions in this case. You can even see this in Apple benchmarks, they do have tons of memory bandwidth esp compared to GPUs but the tokens per second isn't on the same level. SOC type NPUs aren't as powerful as the huge cards/OAM designs and that's really all just down to transistor value. But are they are way more efficient than current CPU instruction sets and that makes them desirable.NPU is not magical, it will steal MemBW from the system.
Yes, it takes the same amount of time with caveats, but you get yourTakes the same amount of time, you only get so many tokens/iterations per second regardless of how you organize the workload. Idk unless like you're running a bunch of tiny models on a huge accelerator but that's not what I'm talking about. Its much better to have models be as big as possible and thats why you don't see anyone commercially trying to run/sell swarms of 4gb distills.
I wasn't talking about the absolute time, but if you have a model that fits in your GPU or CPU if you have good membW, then you can get that almost for free.Would be super nice to get both results at the same time for comparison.
Which tokens per second. The generation usually is matching the available memory bandwidth, the prompt processing sucks due to lack of compute. It also matters at what kind of context size you compare. All popular outlets I have seen that started using LLMs in benchmarks fail to mention this.You can even see this in Apple benchmarks, they do have tons of memory bandwidth esp compared to GPUs but the tokens per second isn't on the same level.
I thought we were comparing to GPUsBut are they are way more efficient than current CPU instruction sets and that makes them desirable.
The difference is not minimal, but the BW difference between PCIe5 and GDDR is so big, that everyone tries to keep the workload on the GPU to avoid transfers. And if you wanted to run inference on NPU fully it would need to have access to full memBW. In other words, NPUs make sense only on chips which have already high memBW, as using NPU on a desktop socket might give you better time to first token but won't give you noticeable increase in token generation over what CPU alone could do. And provided you have dGPU in the system that has enough memory to fit the model, NPU is pointless as offloading prompt processing will run into PCIe 5 bottleneck, unless the model is small enough, but if it is small enough, you can run it on GPUAnd in this day in age, the difference between pcie3 and pcie5 is so minimal for GPUs and other devices even like the 5090, that an SOC class NPU wouldn't make a difference for system BW 99.8% of the time.
If you think millions of people are using LLMs and paying for the privilege to do it, I have a cheap bridge to sell you.Its really the other way around, millions of people are using GPT and other commercially/closed models. Where as most LLM questions/and even lots of image stuff will run just fine on a 16gb GPU from 2020 using distilled models. Meanwhile look at apple AI, they critically depend on GPT simply because they don't have any models or public facing tech of their own.
So in this world where the real OEMs are HP, Dell, Lenovo, ect. And their customers have a pressing need for high bandwidth matrix calculations, and this job can either be done by a giant GPU chip using special GDDR ram, or you can put an NPU into a SOC, save tons of power, use the same ram chips already on the mobo, and still get good enough performance. And if everyone has AI on their machine, it really makes it easy for developers to make AI based apps and programs they can sell to people. This is both microsoft and apple's angle.
Right now it's awful for me to try and run AI on my non-accelerated CPU and I fully have to commit my GPU and lots of power for good performance. Where as NPUs on CPUs really open up the doors to things like games using AI for npcs, or in digital media where the NPU can be used to free up compute for things like effects and transitions. For development it really blows the hinges because it means running one AI into another, locally, suddenly becomes feasible and thats where things really shine. I for one really enjoy using different system prompts on the same model to get more varied results. ie, one programmer bot thinks its a webdev, the other I tell its john carmack, now give me code for X. Would be super nice to get both results at the same time for comparison.
Also look at the GPU market lately, it's not like there's a crypto rush right now, those sales and scalping is legit pressure from the AI market. And if AMD can leverage even a tiny bit of that onto desktop CPU, thats becomes a win for potential volume and margins.
Adobe rents software, they don’t sell it. Apple only has a few paid products, they rent the rest.Don't companies like adobe and apple make a non-significant amount of their money on selling new software?
They are paying for it by giving out priceless information, far more than search engines were gettingIf you think millions of people are using LLMs and paying for the privilege to do it, I have a cheap bridge to sell you.
Not disagreeing with that. 🤣They are paying for it by giving out priceless information, far more than search engines were getting
I did mistype a bit there but I think my point still stands, its their interest to have good creative software. Just because they rent doesn't mean photoshop or final cut is free lol.Adobe rents software, they don’t sell it. Apple only has a few paid products, they rent the rest.
Is that really what you thought I meant? Compared to the nearly 1bn people using openai alone as a huge indicator for the demand of AI? Here in 2025 I'd really think most people would know no product is truly free either.If you think millions of people are using LLMs and paying for the privilege to do it, I have a cheap bridge to sell you.
The highest paid adoption has been in the corporate sector, and that is not even close to “millions”.
If you think millions of people are using LLMs and paying for the privilege to do it, I have a cheap bridge to sell you.
ChatGPT has added 4.5 million paying subscribers since the end of last year, according to The Information. With 20 million subscribers paying for the service, the company is earning “at least $415 million in revenue per month,” a rough estimate that doesn’t account for corporate plans or the new $200 a month Pro tier. The report adds that this could put OpenAI “well within reach” of its revenue projection of $12.7 billion this year.
Then it's not a business. Next.Of course it doesn't mean they are profitable
You mean AMD were not a business while they were leaking money? Waiting for your next stupid 3 words statement. Will that be some 40% silly claim?Then it's not a business. Next.
Yeah they weren't.You mean AMD were not a business while they were leaking money?
The point is simple: ML slop farms do not have a business model to afford all that big iron running 24/7.And that was not my main point anyway.
Just a note. AFAIK YouTube has NEVER been a business because from its inception it has always lost money. Nowadays doesn't lose so much money as before due to Google making all they can do to break even. It was an objective to break even some years ago but they just gave up and didn't talk more about it. They just talk about YouTube revenues but never about profits.Yeah they weren't.
The point is simple: ML slop farms do not have a business model to afford all that big iron running 24/7.
Simple enough for you?
It's an accretive play for GOOG's advertisement business plus it's far, far cheaper than anything ML.AFAIK YouTube has NEVER been a business because from its inception it has always lost money.
Pretty sure they did by making end user experience worse.It was an objective to break even some years ago but they just gave up and didn't talk more about it.
ML big iron is far, far more expensive to buy and operate.Count that with IA could happen something similar.