Discussion RDNA4 + CDNA3 Architectures Thread

[DisEnchantment]

With the GFX940 patches in full swing since first week of March, it is looking like MI300 is not far in the distant future!
Usually AMD takes around 3Qs to get the support in LLVM and amdgpu. Lately, since RDNA2 the window they push to add support for new devices is much reduced to prevent leaks.
But looking at the flurry of code in LLVM, it is a lot of commits. Maybe because US Govt is starting to prepare the SW environment for El Capitan (Maybe to avoid slow bring up situation like Frontier for example)

See here for the GFX940 specific commits

History for llvm/lib/Target/AMDGPU - llvm/llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. - History for llvm/lib/Target/AMDGPU - llvm/llvm-project

github.com

Or Phoronix

More AMD "GFX940" Enablement Work Landing In LLVM - Phoronix

www.phoronix.com

There is a lot more if you know whom to follow in LLVM review chains (before getting merged to github), but I am not going to link AMD employees.

I am starting to think MI300 will launch around the same time like Hopper probably only a couple of months later!
Although I believe Hopper had problems not having a host CPU capable of doing PCIe 5 in the very near future therefore it might have gotten pushed back a bit until SPR and Genoa arrives later in 2022.
If PVC slips again I believe MI300 could launch before it

This is nuts, MI100/200/300 cadence is impressive.



Previous thread on CDNA2 and RDNA3 here

Question - Speculation: RDNA3 + CDNA2 Architectures Thread

Man I have been dying to make this one for a while now. First rumours for RDNA3 are here so new thread time! Just going to start off with this one for now: kopite7kimi on Twitter: "@VideoCardz Ah, I mean a simple mcm design with 10240 cores is not enough. Because the lift from RDNA2 to RDNA3...

forums.anandtech.com

 

[beginner99]

straight up denied AI is useful and thinks the whole thing is a bubble but lol.

To some degree it is a bubble. AI /LLM can be useful in some contexts but these are by now mostly already being used with integrations in office, email and software dev products. once the market has been established and isn't growing exponentially, the demand for new GPUs for AI will go down massively as growth becomes incremental.

 

[SpudLobby]

To be factually correct, there is no such thing as DeepSeek R1 30-32B model. What you are talking about are distills.

Forgot that. Fair enough, I’ve used the LlaMa ones (Haven’t tried Qwen yet). Though to add on, it actually reinforces my point here that they are distilled at the base instead of using V3, because Qwen and LlaMa — LlaMa especially, are just okay to good, not great, it’s just that they’re open and small. But fine tuning them on the reasoning here while prolonging outputs produces vastly better results just not possible at similar scales otherwise. I was just demonstrating the direction of effect here is the opposite of what people expect.

Still think local LLM’s are overrated if not outright hype. Batched inference is vastly more efficient and latency is easily low enough for cloud efficiency & speed gains to be worthwhile, or outright better capability. Spare your RAM, battery life, get real capability and more speed.



In some sense this is a win for AMD among others vs Nvidia in client — most people aren’t going to go crazy for local LLMs. Upscaling tech or other small AI models doing summarization or creative stuff sure, but that’s a meme on memory/compute compared to this.

 

[SpudLobby]

To some degree it is a bubble. AI /LLM can be useful in some contexts but these are by now mostly already being used with integrations in office, email and software dev products. once the market has been established and isn't growing exponentially, the demand for new GPUs for AI will go down massively as growth becomes incremental.

Nonzero bubble quite possible on the hardware end for Nvidia alone if not quite likely — and there may be a 5 steps forward but 1-2 steps back thing at some point, much like dotcom era. It’s not even close to crypto which is scams and speculation.

The statements by those here among other hardware types for years have been totally delusional re: the utility, commercial viability*, etc.

*Like OpenAI lost $5-6B last year per NYT, but with the rate of improvements in cost on the model side they could pretty much double to triple the price of API calls today and increase plus prices and it would still net positive. If the response to this is “well not after DeepSeek V3/R1 is hosted on American hyperscalers at performance (current MS one is like a trial, is slow)” etc, then like, okay. Thanks for demonstrating AI can absolutely be cheap. Just means the hyperscalers will do it and/or models can still improve. It isn’t going anywhere. Nvidia itself I am ambivalent on.

Gemini 2.0 Flash is also utterly dirt cheap, good and Flash Thinking is unlikely to be much different.

 

[coercitiv]

The bubble is not about AI being a scam, it's about companies being overvalued considering many are burning billions while digging for a breakthrough that is supposed to materialize into untold riches. Nvidia will be more than fine, they're selling the shovels.

 

[Win2012R2]

Nvidia will be more than fine, they're selling the shovels

Can Nvidia manage being "just" (for example) 150-200 bln market cap company? They'll probably have to fire 75% of people they employ now, it won't be the same company.

Well, at least we'll be able to get cheap HBM4E^2 gaming GPUs...

 

[Keller_TT]

The bubble is not about AI being a scam, it's about companies being overvalued considering many are burning billions while digging for a breakthrough that is supposed to materialize into untold riches. Nvidia will be more than fine, they're selling the shovels.

AI, to the extent it is being hyped as if it's the greatest thing since the invention of the wheel, and the sheer greed and deception involved in marketing anything with an "AI" sticker, is both a scam and a massively inflated bubble.
GenAI can be optimized for efficiency but in terms of big leaps in performance, we're well into diminishing returns without being able to gather and train data at the scale of the universe and obliterating privacy. I don't expect the US to lead there. May be my most capitalistic country with Direct Democracy with a dedicated canton as Crypto Valley with regulatory freedom for building a decentralized tech stack will do something about it and the EU will flatter us with their imitation.

 

[Heartbreaker]

AI, to the extent it is being hyped as if it's the greatest thing since the invention of the wheel, and the sheer greed and deception involved in marketing anything with an "AI" sticker, is both a scam and a massively inflated bubble.

Whatever level the AI "Bubble" is at, it's deflation to more realistic levels won't harm NVidia. NVidia's inflated stock price is more or less irrelevant to the company. It's not seeking new investment or needing to raise capital. It's a money printing machine.

Unlike Crypto Mining, AI has real value, it's not going to pop like a bubble. It's going to soft land.

 

[basix]

GenAI can be optimized for efficiency but in terms of big leaps in performance, we're well into diminishing returns without being able to gather and train data at the scale of the universe and obliterating privacy.

Well, DeepSeek R1 has just recently shown a new avenue regarding model training, which for the majority relies on synthetic data. And synthetic data can be pushed much more easily towards "the scale of the universe".
The same applies for Chain of Thought. This one primarily scales with compute and not so much with additional data.

And if you follow AI research, there are big leaps each 3 months. Maybe not regarding model accuracy (these take somewhat longer) but regarding model efficiency, RAM consumption and inferencing speed. These are just not as prominent to the public like e.g. a GPT4 Release. But they will be very visible regarding the pricetag of using them.

And there are many, many useful things you can do outside of LLM. Very application specific DNN which yield in huge quality and/or performance gains.

 

[PJVol]

AI, to the extent it is being hyped as if it's the greatest thing since the invention of the wheel, and the sheer greed and deception involved in marketing anything with an "AI" sticker, is both a scam and a massively inflated bubble.

Regarding the topic, I absolutely don't mind if amd (or nvidia) slapping AI stickers literally all over the graphics card, and as long as this doesn't affect the price I'm fine.

 

[Keller_TT]

AI has real value, it's not going to pop like a bubble. It's going to soft land.

It is a bubble when it is unsustainably expensive and energy devouring, and oversold the way it is with a mythical halo, and its limits are soon apparent.
NV of course will more than survive when the market corrects itself, and it would be like dot-com burst for some, and just like being brought down to earth for NV and some others (I'm waiting for it to be the former for a very nefarious unOpen company).

Well, DeepSeek R1 has just recently shown a new avenue regarding model training, which for the majority relies on synthetic data. And synthetic data can be pushed much more easily towards "the scale of the universe".
The same applies for Chain of Thought. This one primarily scales with compute and not so much with additional data.

And if you follow AI research, there are big leaps each 3 months. Maybe not regarding model accuracy (these take somewhat longer) but regarding model efficiency, RAM consumption and inferencing speed. These are just not as prominent to the public like e.g. a GPT4 Release. But they will be very visible regarding the pricetag of using them.

And there are many, many useful things you can do outside of LLM. Very application specific DNN which yield in huge quality and/or performance gains.

I'm not talking just LLM at all. I mean the breadth of GenAI in general, which encompasses a whole lot more. Synthetic data has its limits. I'm talking about real-time, personalized AI, and a number of areas where there is a hard constraint on relevant and quality data.

And you're emphasizing what I just said. DeepSeek R1 has opened eyes in terms of computational efficiency. But that is specific to LLMs and they'll for sure evolve much more. The expectation is other GenAI areas too become much more efficient. That is through both hardware and software architectures, and algorithms. I'm involved with the Swiss AI industry-institute connect who focus on Client and Edge AI too, and a really broad base of what AI can bring without imposing itself. So, I know where the limits are, and the political implications that can't be overlooked.

 

[Keller_TT]

Regarding the topic, I absolutely don't mind if amd (or nvidia) slapping AI stickers literally all over the graphics card, and as long as this doesn't affect the price I'm fine.

Is it not an oxymoron in How It Works™?

 

[basix]

And you're emphasizing what I just said. DeepSeek R1 has opened eyes in terms of computational efficiency. But that is specific to LLMs and they'll for sure evolve much more. The expectation is other GenAI areas too become much more efficient. That is through both hardware and software architectures, and algorithms. I'm involved with the Swiss AI industry-institute connect who focus on Client and Edge AI too, and a really broad base of what AI can bring without imposing itself. So, I know where the limits are, and the political implications that can't be overlooked.

Well, hi then to a fellow countryman.

I am working in engineering and AI has so many use cases and benefits without really compromising privacy. There it is mostly about physics & chemistry (which have no privacy) and coding (which is more a licensing topic regarding training data).
It really depends on the use case. Everything connected and related to people is much more nuanced and complicated (social and political implications as you say). Medical data is a good example for that. Potentially very useful, but with huge concerns on the other side.
And regarding leaps and breakthroughs it is always a question, how much money is spent and how many bright people are working on it. And for many very specific things there might be a breakthrough but you simply do not take notice of it (because not on your radar, so many of them).

I fully agree with you that AI as term is hugely inflated and overhyped. The same applies for respective stock market assets, Nvidia is just one of many.
Nevertheless: AI has very nice use cases and benefits. It will stay. It will get better. I said already 2017 or so that AI will be the next big thing. And here we are, much has changed. Much has improved. Much has still not been achieved yet (fully self-driving cars anyone?).

 

[Keller_TT]

Well, hi then to a fellow countryman.

Grüezi 👋

It will get better. I said already 2017 or so that AI will be the next big thing. And here we are, much has changed. Much has improved. Much has still not been achieved yet (fully self-driving cars anyone?).

Well, that's a whole new can of worms with "fully self-driving cars". Apart from feasibility in cities, I know there are a number of legal hurdles about liabilities and the rules based on which self-driving cars might decide life and death decisions.
That belongs to the application of deontic logic, and I learned about some truly fascinating research into it from a source that would blow your mind: A particular branch of exegesis of ancient Indian sacred compositions called Mimamsa and its sub-school that went to absurd lengths on linguistics and determining injunctions and morality with which it had to be approached. The Indians kicked ass when it came to linguistics and analytic philosophy, and I know there's a paper that's popular among Indians in the AI field from an old NASA scientist about how Sanskrit was the world's first generative grammar that was specifically adapted for knowledge representation, and has a body of literature that far exceeds that of Greek and Latin combined.
Prof. Agata Ciabattoni of TU Wien has gone deep into that area with applicability to self-driving cars here:

reposiTUm: Disambiguating Permissions: A Contribution from Mimamsa

repositum.tuwien.at

Reasoning Tools for Deontic Logic and Applications to Indian Sacred Texts

mimamsa.logic.at

I attended an AI workshop there that was fully organized around this, and we had some truly astounding discussions.

 

[poke01]

7900XTX sold out, all of them in Australia due to failure of RTX 50

I'll be getting the 9070XT day 1

 

[gdansk]

I'll be getting the 9070XT day 1

Don't be so confident. I think AMD had originally intended this as a small launch.

 

[poke01]

Don't be so confident. I think AMD had originally intended this as a small launch.

you have to buy day 1, thats the only way. Otherwise the price will increase and/or stock will take months to recover

 

[gdansk]

you have to buy day 1, thats the only way. Otherwise the price will increase and/or stock will take months to recover

Two problems:
There will be few to buy on day 1.
Is it even a substantial upgrade from a 4070 Super?

 

[poke01]

Is it even a substantial upgrade from a 4070 Super?'

If its on par with a 4080 super and under $650 yes. Considering the next gen won't come till 2027.

maybe its FOMO lol

 

[gdansk]

If its on par with a 4080 super and under $650 yes

And I doubt that too. If 7900 XTX are selling out now for much more, what incentive do they have to price it like that? So it's probably quite a bit slower.

 

[branch_suggestion]

And I doubt that too. If 7900 XTX are selling out now for much more, what incentive do they have to price it like that? So it's probably quite a bit slower.

There used to be a standard called 30% perf/$/gen.
If AMD wants market share they may indeed target such a value, having rough software parity would help too.

 

[RnR_au]

And I doubt that too. If 7900 XTX are selling out now for much more, what incentive do they have to price it like that? So it's probably quite a bit slower.

AMD said that they liked what they saw at the 7900GRE and 7800XT pricing points.

 

[beginner99]

GenAI can be optimized for efficiency but in terms of big leaps in performance, we're well into diminishing returns without being able to gather and train data at the scale of the universe and obliterating privacy

I'm firmly in the "AutoGPT" camp. the advantage of what was developed is to be able (not always but pretty well) understand what the user is asking / demanding. What is less useful is what is being done with that capability.

The productivity boost will come when AI can do automation on text prompt / voice prompt. Like at work the ordering process is a complete and utter shit show. The people that have to use it regularly tend to need like 10 minutes per simple order. because it's slow and often fails and has to be tried multiple times. Imagine you can just tell an AI to order for you in the internal system. This of course is not plug and play anymore and will require software development so not so easy and cheap to implement but that would actual give productivity benefits.

 

[RnR_au]

Nvidia will be more than fine, they're selling the shovels.

Nvidia is selling the shovels most folk are using in training the models. Anyone can build shovels for inference. The former as a market will be much smaller than the latter once inference really takes hold in the economy and its becomes normalised across all industries.

 

[Tuna-Fish]

Don't be so confident. I think AMD had originally intended this as a small launch.

But they have had at least an extra month to ship out supply.

 

[adroc_thurston]

I think AMD had originally intended this as a small launch.

No. But they didn't expect it to be a winning product either.