Discussion Speculation: Zen 4 (EPYC 4 "Genoa", Ryzen 7000, etc.)

[Vattila]

Except for the details about the improvements in the microarchitecture, we now know pretty well what to expect with Zen 3.

The leaked presentation by AMD Senior Manager Martin Hilgeman shows that EPYC 3 "Milan" will, as promised and expected, reuse the current platform (SP3), and the system architecture and packaging looks to be the same, with the same 9-die chiplet design and the same maximum core and thread-count (no SMT-4, contrary to rumour). The biggest change revealed so far is the enlargement of the compute complex from 4 cores to 8 cores, all sharing a larger L3 cache ("32+ MB", likely to double to 64 MB, I think).

Hilgeman's slides did also show that EPYC 4 "Genoa" is in the definition phase (or was at the time of the presentation in September, at least), and will come with a new platform (SP5), with new memory support (likely DDR5).

What else do you think we will see with Zen 4? PCI-Express 5 support? Increased core-count? 4-way SMT? New packaging (interposer, 2.5D, 3D)? Integrated memory on package (HBM)?

Vote in the poll and share your thoughts!

 

[moinmoin]

So if that's ruled out, then I agree with you. Maybe we'll see >1 layer of V-cache on some of the SKUs.

Infinity Cache != 3D V-Cache

On Ponte Vecchio Rambo Cache tiles appear to be used as interposers under the central tiles, connecting them all. Cache connecting GPU MCM chiplets would be the next logical step for Infinity Cache as used in RDNA2. Looks like both Genoa and RDNA3 will take that step at once.

 

[Mopetar]

Milan will continue to soak up this capacity, at increasing pace.

Given the dollar amount and span of time attached to the deal it's hard to imagine that it is for Milan.

There is a lot of talk about Active Silicon Bridges - related to RDNA3 - which could be stacked.

If this technology makes it to the future generations of Zen, the connection between CCD and IOD could be stacked active silicon bridge. And for that, TSMC N6 based IOD would be needed.

It seems unlikely that this is happening with Zen 4 or we'd have heard about it by now. Any leak/rumor for Zen 4 to this point has had a separate IO die, so if this were to happen at all, it's probably not for several years at the soonest.

And from Charlie's article, A LOT of stacked L3 will be needed in the upcoming server processors.

That can be done on the chiplets. Moving the L3 to the IO die just introduces a lot of additional latency for not much additional capacity over having L3 on chiplet and the option of an additional stacked layer on top of each chiplet. Any cache on the IO die is going to be an L4/SLC or even just some glorified buffer for memory accesses as opposed to an extension of L3 cache.

 

[Joe NYC]

Given the dollar amount and span of time attached to the deal it's hard to imagine that it is for Milan.

There were some rumors that Tesla car info / entertainment system might be using Global Foundries.

AMD also has a growing business for embedded devices, many of which could also use Global Foundries.

It seems unlikely that this is happening with Zen 4 or we'd have heard about it by now. Any leak/rumor for Zen 4 to this point has had a separate IO die, so if this were to happen at all, it's probably not for several years at the soonest.

IO die would be separate die in any case, just a question of how it is connected to CCDs.

Zen 4 is an "old" design at this point, before AMD had any experience with SoIC and hybrid bond stacking.

There will be Zen4c - a new design - which can incorporate a few changes.

Without going into details of Charlies article, the implication of what he is reporting on is that the server chips for hyperscalesrs will need A LOT of cache.

And then, suddenly, there is a core also targeted for cloud hyperscalers. So, to re-iterate my theory, Bergamo could move beyond Genoa in packaging, in internal arrangement of components, and could move to the rumored Active Silicon Bridges, to ditch potentially ditch the fan out and SerDes.

That can be done on the chiplets. Moving the L3 to the IO die just introduces a lot of additional latency for not much additional capacity over having L3 on chiplet and the option of an additional stacked layer on top of each chiplet. Any cache on the IO die is going to be an L4/SLC or even just some glorified buffer for memory accesses as opposed to an extension of L3 cache.

The point of the Active Silicon Bridge that would also include SRAM is that the latency should be in the ballpark of V-Cache.

IOD would just be the other side of the bridge, with stacked SoIC hybrid bond connection, for which TSMC N6 would be probably necessary.

Another advantage of the Active Bridge connection is that latency to L3s of other chiplets would be extremely low, making L3 behave as if it was a SLC.

So, I would not expect this for Genoa but maybe for Bergamo.

 

[DisEnchantment]

Maybe I am misunderstanding the terms but to my understanding it is the following:
Fanout is just a technology to adapt a high density bump pitch (die) to a lower density bump pitch (package, interposer, etc.). As I said, just to prevent pad limitation.
It depends on what you connect to the fanout. If it is the same old organic package then power efficiency won't change. But you could also connect it to an interposer, LSI, EFB, EMIB, etc. This of course would improve power efficiency. But from everything we know I think it will still be the organic package.
If I am wrong please feel free to correct me.

Fan Out is a generic term applied to the process of enlarging the structure around a die using some filler material.
This filler material adds many properties to the original die which improves its CTE, mechanical/warpage characteristics, etc. besides providing a way of embedding passive RDLs which can be used to reroute higher density bumps to lower density bumps or to encapsulate active dies which allows to use repeaters/switches for low power high density parallel interconnects like HBI . Or you could just have plain high density metal traces for SerDes providing higher density interconnects without going through the substrate.
The final outcome at the end of fan out packaging is a bigger die with all the elements inside which is easier for assembly and mechanically/thermally stronger.
In short, fan out doesn't mean anything without knowing what's inside it. A fan out package could just contain passive interconnects, or just active interconnects, or logic, or all of them.

Think of fan out as epoxy resin glue tying together many pieces of wood inside a single block, once harden it becomes a single structure and holds all the interconnects/chiplets/cu pillars/logic/etc inside in place with all said benefits above.

Kinda. Doesn't support CXL.mem, so AMD will probably be first to that. Which is really embarrassing for Intel, having invented it and all.

You don't have to necessarily look at it that way, Intel may have invented it, but AMD had some head start having implemented it in IF/Trento (and possibly CCIX according to initial Genoa roadmap). They just ported the "IF logic" to CXL. IF 3.x can do what CXL.cache/mem can do.
But Intel and AMD standardizing on it will be good for the whole industry.

 

[NostaSeronx]

There were some rumors that Tesla car info / entertainment system might be using Global Foundries.

Which happened to be just a Picasso-e variant for now.
YE1807C3T4MFB (Raven-H 35W-54W) -> YE180FC3T4MFG (Picasso-H 35W-54W)

AMD also has a growing business for embedded devices, many of which could also use Global Foundries.

If it is FinFETs, then it would be very brief.

Fab8 based on internal sources:
1. About to be shuttered or sold, related to Fab 8-Module 2 being canned. (FinFET group employee)
2. About to kill off FinFET nodes for 90nm-45nm SOI nodes (worst case build up of GlobalWafers MEMC SOI). (45nm RF/FD group employee)
3. About to kill off FinFET nodes for 90nm-45nm-22nm-12nm SOI nodes (best case build up of GlobalWafers MEMC SOI). (Dresden/Singapore FDX group employee)

Basically, GlobalFoundries can't hide negative income by Fab8 from investors much longer.
50K-wpm in 2017 w/ high optimism to ~15K-wpm in 2021 w/ extremely low optimism of turning around FinFETs.

Going forward =
No new FinFET products whatsoever at GlobalFoundries.
1. No new capacity, while pushing for less capacity to fit in more mature nodes.
2. Unreliable single module, GloFo path forward is node sharing across all three fabs starting with:: "Our worldwide manufacturing footprint gives us the ability to produce what our customers need on more than one continent. For example, Fab 7 is transferring certain 55nm and 40nm technology nodes to GF’s Fab 1 in Dresden, Germany, so that customers will have two sources of supply for them from within GF, helping to ensure business continuity." - https://gf.com/blog/inside-look-gfs-fab-7-singapore
3. Lack of customers willing to stay locked in Fin w/ no new node or increased capacity at GloFo. Especially, when Intel/Samsung/TSMC is better, or within specific regions like SMIC/HLMC.

Crossover for Fin(1x to 0x) to FDX(0x to 1x) is set for 2H'23 at Malta, with the current schedule. Anyone doing non-compatible 14LPP designs will be shooting themselves in the foot.

 

[eek2121]

How in the world do you reason this?

At the rate they need silicon to continue rapid growth, using TSMC for everything will probably hurt them more. Don't forget that Nvidia is returning and Intel is also competing for the latest nodes. 33-40% of your silicon area for servers from GloFlo will help a lot. One of the big advantages of chiplets is to do precisely this sort of mixing to maximize marketshare & margins.

Moving everything to TSMC really cuts in to how much they can produce and in a market where AMD has the best products that's far worse than making a marginally better product.

There are limits to how much an IO die can shrink and these are large chips, especially for their server parts.

It also ignores all of the AMD commitments to buy wafers from GF that they've agreed to (and recently expanded) with Global Foundries.



There isn't a lot that can be stacked on an IO die and even if you could drop a massive L4 cache layer on top of it, why do that instead of stacking additional cache on the Zen chiplets? The added latency of having to go to the IO die for that cache is going to limit the actual performance gains it can achieve. Zen 3D has shown us that the set of applications that benefit most from the larger cache are limited, so how much an L4 adds outside of niche uses is questionable.

3D stacking on the IO die wouldn't be available if they used GloFo. There are other various packaging technologies that would also not be available. It would be more expensive and would take longer to package the chip together if they have to import chips from GloFo.

TSMC has PLENTY of 6nm capacity. 6nm is 7nm with EUV added. AMD doesn't have issues keeping Zen 3 in stock, they most certainly won't have issues with a 6nm IO die and 5nm compute chiplets.

Regarding the wafer commitment, Zen 3 and Milan/Milan X will still be in production well after Zen 4 launches. Both still use 12/14nm IO dies. I also suspect we'll see lower end Zen 3 parts release once Zen 4 launches. Maybe a 3500x/3300x equivalent. The 3800X3D will also remain on sale. They may also use 14nm/12nm for other projects we aren't yet privy to. The only way I could see AMD going with GloFo for IO dies is if they get 12LP+ at an extremely cheap per-wafer price. 12LP+ only has about a 15% area improvement over 14nm/12nm. The Zen 4 IO die would be > 15% larger at 14nm due to supporting PCIE5, DDR5, a GPU, etc.

EDIT: For reference, TSMC process comparison here: TSMC Reveals 6 nm Process Technology: 7 nm with Higher Transistor Density (anandtech.com) oh and Zen 3 Threadripper hasn't even launched yet. That will also use GloFo for the IO die.

AMD should push up their Zen 4 release date. They are giving Intel way too much time for a secondary strike. Let's assume that Zen 4 is a great CPU with huge IPC improvements over Zen 3. Then Intel responds with something beyond Zen 4 by the end of the year that is 10-15% better than Zen 4? What would AMD do?

The sooner AMD releases Zen 4, the more time they have to sell the CPU before Intel can respond with something better.

I look at CPU prices for Alder Lake and cannot figure out what AMD is thinking.

Why? They have no issues selling everything, and Intel is not a compelling option for most.

 

[Mopetar]

There were some rumors that Tesla car info / entertainment system might be using Global Foundries.

AMD also has a growing business for embedded devices, many of which could also use Global Foundries.

Still too many wafers for both of those together.

Zen 4 is an "old" design at this point, before AMD had any experience with SoIC and hybrid bond stacking.

This is the Zen 4 speculation thread. Why talk about what might be for Zen 6 or 7 in this thread?

Without going into details of Charlies article, the implication of what he is reporting on is that the server chips for hyperscalesrs will need A LOT of cache.

Still better to put it on the chiplets. Doing multiple layers of cache on chiplets still makes more sense than moving L3 cache to the IO die.

3D stacking on the IO die wouldn't be available if they used GloFo. There are other various packaging technologies that would also not be available. It would be more expensive and would take longer to package the chip together if they have to import chips from GloFo.

They aren't stacking on the IO die for Zen 4. That whole line of argument is irrelevant for this product.

Also the packaging isn't done in Taiwan so the chips would get shipped off elsewhere regardless.

 

[Joe NYC]

This is the Zen 4 speculation thread. Why talk about what might be for Zen 6 or 7 in this thread?

My last sentence was that it more likely come with Bergamo, not Genoa, both of which are Zen 4 generation.

Still better to put it on the chiplets. Doing multiple layers of cache on chiplets still makes more sense than moving L3 cache to the IO die.

"Bridge" has 2 ends, and one end would still be CCD, the other end would be IOD.

So, it is not "moving" L3. It's just that the stacked L3 die would have other functionality as well - connecting dies. Which is what the rumored MCD of RDNA3 might do.

Also the packaging isn't done in Taiwan so the chips would get shipped off elsewhere regardless.

EPYC packaging supposedly is done in Taiwan. I don't have a link, I just recall reading it.

 

[lobz]

AMD should push up their Zen 4 release date. They are giving Intel way too much time for a secondary strike. Let's assume that Zen 4 is a great CPU with huge IPC improvements over Zen 3. Then Intel responds with something beyond Zen 4 by the end of the year that is 10-15% better than Zen 4? What would AMD do?

The sooner AMD releases Zen 4, the more time they have to sell the CPU before Intel can respond with something better.

I look at CPU prices for Alder Lake and cannot figure out what AMD is thinking.

There is no secondary strike for quite a while after Zen 4 comes out. AMD simply chooses to give Intel the low-end and mid tier of the market right now, every other step would mean lower ASP and lower operating income, given that every single produced wafer is fully sold out immediately.

 

[inf64]

Zen4 will utterly destroy ADL and RocketLake across the board, it won't be even funny. I expect them to launch in Q3 with solid volume.

 

[TESKATLIPOKA]

Zen4 will utterly destroy ADL and RocketLake across the board, it won't be even funny. I expect them to launch in Q3 with solid volume.

Rocket Lake? Didn't you mean Raptor Lake?
It's nice having high expectations, but tone down your hype, please. We still don't know how powerfull Zen 4 will be.
At least wait for reviews to be out to be sure.

 

[lobz]

Zen4 will utterly destroy ADL and RocketLake across the board, it won't be even funny. I expect them to launch in Q3 with solid volume.

More like December, with volume maybe in January some time.

 

[scineram]

There were some rumors that Tesla car info / entertainment system might be using Global Foundries.

Tesla uses Picasso, so there.

 

[Hitman928]

TSMC has PLENTY of 6nm capacity. 6nm is 7nm with EUV added. AMD doesn't have issues keeping Zen 3 in stock, they most certainly won't have issues with a 6nm IO die and 5nm compute chiplets.

The problem as I see it is that AMD will continue to use a lot of 7/6 nm wafers for compute/bridge/cache dies. Plus the console chips aren't moving to 5 nm so they are taking up a significant amount of 7/6 nm wafers as well. I don't expect AMD to be able to make enough 5nm and 7/6nm products even if the IO die stays on GF. Moving the IO die to 7/6 nm will only further constrain their ability to supply their customers. Maybe its worth it if AMD can significantly improve efficiency by moving the IO die to TSMC to keep a clear premium tier product lead but I have my doubts that moving the IO die makes that much of a difference. Whatever process it is on, I hope someone does a deep dive into the new IOD and how it compares to the old one.

 

[MadRat]

What would the mainboard contain in support of Zen4? For people willing to spend $1,000 on a CPU, and $1,000 for memory, and $1,000 of storage, and a $1,000 for graphics... What would be the point of $200 mainboard? Just seems like you would want at least $1,000 in a mainboard, too.

 

[Markfw]

What would the mainboard contain in support of Zen4? For people willing to spend $1,000 on a CPU, and $1,000 for memory, and $1,000 of storage, and a $1,000 for graphics... What would be the point of $200 mainboard? Just seems like you would want at least $1,000 in a mainboard, too.

Thats just stupid.... Even a dual socket EPYC board is only about $650. The top of the line Alder lake and Ryzen motherboards are $600. A CPU and video card are a lot more complex than a motherboard. Mmeory is only about speed and quantity, it would also not be $1000. Your post makes no sense, and is pointless.

 

[Mopetar]

My last sentence was that it more likely come with Bergamo, not Genoa, both of which are Zen 4 generation.

Still seems far too ambitious for a product that will be out in 2023. They've just started learning how to get a layer of cache on these chips. It's a big step beyond that for what you're proposing.

So, it is not "moving" L3. It's just that the stacked L3 die would have other functionality as well - connecting dies. Which is what the rumored MCD of RDNA3 might do.

Have a link to the description of this technology? I'm not sure what you're envisioning this would look like, but they way you're describing it and I interpret that makes me feel as though there's a misunderstanding of the technology in there.

Unless they're going to invert the layers and put the cache on the bottom layer I'm not sure why any of the transistors for connecting to any kind of bus would be put on a top layer with the cache.

EPYC packaging supposedly is done in Taiwan. I don't have a link, I just recall reading it.

Malaysia, at least for the first Epyc CPUs.

I couldn't find a clear picture of a Milan CPU, but one that I looked at on eBay appeared to still have Malaysia as the country where final packaging was done.

 

[Joe NYC]

Still seems far too ambitious for a product that will be out in 2023. They've just started learning how to get a layer of cache on these chips. It's a big step beyond that for what you're proposing.

Have a link to the description of this technology? I'm not sure what you're envisioning this would look like, but they way you're describing it and I interpret that makes me feel as though there's a misunderstanding of the technology in there.

A lot of this speculation comes from people expect RDNA3 to be. Navi31 and Navi32 are supposed to have 2 GPU modules with abbreviation "GCD" and one connecting die named "MCD".

This is the best picture I could find from patent applications:

https://www.freepatentsonline.com/y2021/0098419.html

So we will find out what is possible, what AMD has been up to, when RDNA3 is announced. Since Bergamo is after RDNA3, the technology that RDNA3 uses should also be available to Bergamo.

 

[moinmoin]

I couldn't find a clear picture of a Milan CPU, but one that I looked at on eBay appeared to still have Malaysia as the country where final packaging was done.

It's still "Made in Malaysia" indeed (timestamp 14:39):

 

[Timorous]

A lot of this speculation comes from people expect RDNA3 to be. Navi31 and Navi32 are supposed to have 2 GPU modules with abbreviation "GCD" and one connecting die named "MCD".

This is the best picture I could find from patent applications:

View attachment 56975

https://www.freepatentsonline.com/y2021/0098419.html

So we will find out what is possible, what AMD has been up to, when RDNA3 is announced. Since Bergamo is after RDNA3, the technology that RDNA3 uses should also be available to Bergamo.

From what I have read N31 and N32 will have multiple MCDs to make up the desired cache amount. Makes more sense, especially if they happen to be the same dies that get stacked on Milan-X / Ryzen 3D.

 

[amd6502]

Yeah, zen 4 needs ddr5, doesn't it? And 12LP+ and 22FDX don't have that yet.

I thought the difference between DDR4 and DDR5 is very very minor.

 

[Exist50]

I thought the difference between DDR4 and DDR5 is very very minor.

From an underlying technology standpoint, DDR5 is a huge change, and will scale to much higher bandwidth with DDR4 (more important for server than client). Its only real weakness right now is the dearth of low-latency kits (i.e. what gamers like), but that's something that happens at the beginning of every DDR generation.

 

[Joe NYC]

From what I have read N31 and N32 will have multiple MCDs to make up the desired cache amount. Makes more sense, especially if they happen to be the same dies that get stacked on Milan-X / Ryzen 3D.

I also heard that, that there may be multiple MCD dies.

But I don't think these will be the same as the V-Cache in Milan-X or Ryzen 3D, because this die will have additional functionality.

 

[amd6502]

The IO die will be mostly IO which by its very nature isn't conducive to shrinks as physical interfaces are hard to make smaller. For that matter, does GloFo support fan-out packaging on its advanced nodes? That's the only way I can see them offering a more featured IO die at less area.

With the above in mind, would shrinking IO-die to say 6/7nm possibly make more sense if they added a small iGPU and/or and L4 cache and kept the IO mostly near the perimeter?

Nosta I think would suggest not shrinking but going FDSOI for the IOX.

 

[jamescox]

I also heard that, that there may be multiple MCD dies.

But I don't think these will be the same as the V-Cache in Milan-X or Ryzen 3D, because this die will have additional functionality.

512 MB as 64 MB die would be 8 die which is the same number of CPU die as in Bergamo. It would sit on top of or under the IO die and overlap the cpu chiplets to act as a bridge and as cache die. I was hoping for 1 GB of cache in Bergamo though, either as L3 or L4. Perhaps we will get an increase up to 128 MB for some die or it will be more than one stack thick. If they can go up to 4 high, then it could have 2 GB. With the TSV connections it could act as L3 with significantly faster connection to the rest of the system. It really would look a lot more like a monolithic L3 cache. It would fit in with the lower power consumption goals of Bergamo since there would be no serdes links to cpu chiplets. The TSV connections would be much lower power and significantly higher bandwidth.

It would also allow massive caches without using any extra 2D packaging area since it would be stacked. The entire IO die plus 8 cpu chiplets might actually fit in a single reticle size. The IO die would be smaller than the Genoa die since it would have half of the number of serdes links. The space for TSVs is likely much smaller than serdes links that can operate at pci-e 5 speeds or greater. The cpu chiplets would be similar in size even with 2x cores since cache would be stacked and possibly higher density, lower power libraries/process.

They definitely could use the same cache chip across several products. The TSV area isn’t that large and they could do something like connections in the middle for acting as v-cache and connections on the ends when acting as a bridge.

I don’t know how this fits with the RDNA3 GPUs, which are (I believe) rumored to have 256 and 512 MB variants. With 2 graphics die and 4 bridge/cache chips, that would be 256 MB. There likely would not be room for 8 cache die between 2 graphics die, so that would indicate that the cache chips are 128 MB, either larger die or 2 stacks. The 64 MB die is only 36 mm2, so going with a 128 MB die at around 72 mm2 (roughly same size as cpu chiplet) would make a lot sense. I don’t know how they would reuse those as v-cache chips though, since it would cover the whole cpu chiplet. That would possibly cause thermal issues unless the cache die moves to the bottom of the stack with the cpu on top.