Discussion ITT We speculate on IMFT 96L 3D NAND (and beyond)

[cbn]

According to Techinsights here IMFT 64L NAND is two 32L 20nm 3D NANDs string stacked.

Current IMFT "dies" for 64L are 256Gb 20nm TLC (ie, 2 x 128Gb 32L TLC string stacked), 512Gb 20nm TLC (ie, 2 x 256Gb 32L TLC string stacked) and 1024Gb 20nm QLC (ie, 2 x 512Gb 32L QLC* string stacked). This is the second generation of 3D NAND from IMFT. The first generation comprised of a single 32L die which could used as either 384Gb TLC or 256Gb MLC.

*These 32L 512Gb QLC could hypothetically also be used as 384Gb TLC or 256Gb MLC (essentially what first Gen IMFT die was or is).

Next generation of IMFT NAND (the third generation) is 96L but don't yet have specs for the die capacity.

My speculation for third Gen (3 x 32L string stacked) is the IMFT will keep the layers per die at 32, but will use 16nm lithography rather than 20nm. This will result in an increase in capacity for each 32L die from 384Gb TLC to 512Gb TLC. String stack three of these 32L 512Gb TLC dies together and it comes out to be either 96L 1536Gb TLC or 96L 2048Gb QLC. So although layers only increase 50% (comparing 3rd Gen to 2nd Gen) 3D QLC capacity per "die" increases by 2x.

So that would be the large 96L die for 3rd Gen....but next question is whether or not there will be a smaller die for 3rd Gen? I am not sure...but it would make sense there would be at least one. Maybe 3 x 32L 256Gb TLC for 96L 768Gb TLC or 96L 1024Gb QLC? (This smaller 96L 1024Gb QLC for increased parallelism (and improved yields) over the 96L 2048Gb QLC.

P.S. For fourth Gen Intel and Micron are going there separate ways but my speculation is that the base 32L die (now shrunk down to 16nm) would increase to 64L. So fourth Gen could double capacity over third Gen by increasing layers to 192L (via 3 x 64L 16nm 3D NAND string stacked).

 

[cbn]

Glaring_Mistake points out in this post that Intel has both 20nm and 16nm listed for 1st Gen 3D NAND.

My speculation is that 16nm was being used for Data center 3D MLC as a warm up for 16nm 3D TLC and 16nm 3D QLC. (As yields at the fab level increased* 16nm 3D TLC and 16nm 3D QLC would follow).

*Improved error correction on future SSD controllers would no doubt help as well.

 

[Billy Tallis]

My speculation for third Gen (3 x 32L string stacked) is the IMFT will keep the layers per die at 32,

String stacking does not mean taking multiple independent dies and making a sandwich out of them. It's still built as one die with just one set of peripheral circuitry under all the layers of NAND. String stacking is used to avoid having to make high aspect ratio holes through a very tall stack of layers, by instead doing deposition and etch for the first half of the layers, and then going back and doing it again for the second half. This significantly slows down manufacturing and requires making sure the two sets of layers are aligned so that the corresponding vertical strings of memory cells are actually connected.

Going to 3x string stacking would make that manufacturing throughput penalty even worse, and increase the yield impact of any alignment imperfections. Intel and Micron are probably going to avoid that for as long as possible, using other techniques to increase the practical layer count in a single string, as the rest of the industry has been doing. IMFT 96L is officially 48+48, not 32+32+32; this was shown at FMS 2018.

 

[cbn]

IMFT 96L is officially 48+48, not 32+32+32; this was shown at FMS 2018.

Thank you for the information!

https://www.anandtech.com/show/13184/flash-memory-summit-2018-micron-keynote-live-blog-qlc-flash

String stacking does not mean taking multiple independent dies and making a sandwich out of them. It's still built as one die with just one set of peripheral circuitry under all the layers of NAND. String stacking is used to avoid having to make high aspect ratio holes through a very tall stack of layers, by instead doing deposition and etch for the first half of the layers, and then going back and doing it again for the second half. This significantly slows down manufacturing and requires making sure the two sets of layers are aligned so that the corresponding vertical strings of memory cells are actually connected.

.....and correction on how the dies are built.

 

[cbn]

Going to 3x string stacking would make that manufacturing throughput penalty even worse, and increase the yield impact of any alignment imperfections. Intel and Micron are probably going to avoid that for as long as possible, using other techniques to increase the practical layer count in a single string, as the rest of the industry has been doing.

It seems to me this would have to happen with fourth Gen (via 3 x 64L string stacking) if Intel and Micron want to keep the same rate of density increases.

1st Gen ---> 2nd Gen = doubling capacity per die (for the largest 2nd Gen die)*.
2nd Gen ---> 3rd Gen = doubling capacity per die (for the largest die assuming IMFT does use 16nm)

*largest capacity die for 2nd Gen is 1024Gb QLC which also works out to be 768Gb TLC (768Gb TLC is 2x the capacity of the 1st Gen 384Gb TLC die)

Either that or tools allowing 96L arrive and then fouth Gen could be 192L via 2 x 96L string stacking.

With this noted, I wonder what happens at fifth Gen? 288L via 3 x 96L string stacking + half node lithography shrink (ie,14nm)?

 

[cbn]

With Intel focused on the Ruler form factor I wonder if they decide to increase maximum die size for NAND. This to make use of the wider form factor of Ruler (In contrast, M.2 is only 22mm wide) and therefore maximize the capacity of each Ruler SSD.

P.S. With Ruler form factor being so long having decreased parallelism per GB (because of the big die) shouldn't be a problem because there is so much space on the Ruler factor for plenty of the big packages. With this noted, I do think another interesting usage for big die NAND could also be DIMMs (or SO-DIMMs) which are also significantly wider than M.2. (E.g. NVDIMM-P that use Optane and NAND integrated at the firmware level). With this noted, I do think Optane and NAND integration (at the firmware level) could happen first with the Ruler SSD (via 1.)usage of Optane to replace the very large amount of DRAM buffer needed + 2.) Optane for cache, probably read only......but potentially also for write if enough Optane is present) then this Optane/NAND firmware level integration progresses over to DIMMs (via NVDIMM-P with both Optane and NAND).

 

[cbn]

After doing the Sequential write comparison in this post I am now convinced that 96L IMFT will include a 1536 Tb TLC die (which converts to a 2048Gb QLC die).

And this die is currently being used in the 960GB Crucial BX500.

 

[cbn]

Here is recent announcement about Silicon Motion's SM3282:

http://www.siliconmotion.com/A6.1.Detail_News.php?sn=255

Notice the following statement:

The new SM3282 with on-chip USB 3.2 Gen 1 interface provides a complete single-chip hardware and software solution supporting the USB Attached SCSI Protocol (UASP) and 2 NAND channels with 4 CE (Chip Enables) per channel, enabling up to two terabytes of storage with the latest generation of 96-layer QLC NAND.

2TB from 8 chips means each 96L die is 2048Gb 3D QLC.

With Silicon Motion working closely with IMFT + the estimate I linked in the previous post I'm guessing Silicon Motion is referring to 2048Gb 96L 3D QLC made by IMFT.

Assuming this is indeed IMFT NAND and it is made on 3D 20nm the die size would be ~230 mm2 (IMFT 20nm 128Gb planar MLC was 202mm2 as a reference point).

OK so if IMFT 3rd Gen 96L does have this 2048Gb 3D QLC die where does Intel and Micron go from there? (Remember after third gen Micron and Intel will develop NAND separately).

EDIT: Here is an excerpt from the Tweaktown 960GB BX500 review.

The 96L memory does increase the number of die per wafer (the die is smaller), but we don't know how many more per wafer.

^^^^ Going by that info what I think is a 96L 1536Gb 3D TLC/2048 3D QLC die would be using 16nm.

 

[cbn]

So with Intel/Micron using 16nm 3D NAND perhaps 14nm for 3D NAND is next for either Intel or Micron? This initially for TLC and then eventually QLC when enough controller ECC and/or 3DXpoint integration is available?

NOTE: For Planar NAND IMFT stopped at 16nm, but SK Hynix and Samsung did go to 14nm for planar NAND (SK Hynix used 14nm both MLC and TLC).

P.S. The first SSD (as mentioned earlier in the thread) to use 16nm IMFT 3D NAND was the Intel DC S3520: (It used 16nm Gen 1 3D NAND as MLC):

https://ark.intel.com/content/www/us/en/ark/products/series/93103/intel-ssd-dc-s3520-series.html

 

[cbn]

Very interesting that with AMD EPYC Server OEMs can increase long ruler SSDs per 2U by 68%:

https://www.anandtech.com/show/1449...d-epycbased-system-with-108-intels-ruler-ssds

Xeon Scalable CPUs have 48 PCIe lanes, whereas AMD’s existing EPYC processors feature 128 PCIe, AMD’s platforms can actually take more advantage of such SSDs than Intel’s own platforms.

It accomplishes this by using the back of the server to mount long ruler SSDs:

(I reckon this is good news for Intel's NAND development team)

 

[cbn]

Some details on Intel's Long Ruler form factor SSD:

https://www.intel.com/content/www/u...y-storage/solid-state-drives/edsff-brief.html

EDSFF drives were designed to optimize capacity per drive. With 36 media sites on the E1.L this drive can scale to higher capacities without expensive and complex die stacking.

The E1.L is up to 2 times more thermally efficient than U.2 15mm drives

And here is some info on bandwidth:

https://www.anandtech.com/show/11702/intel-introduces-new-ruler-ssd-for-servers

The initial ruler SSDs will use the SFF-TA-1002 "Gen-Z" connector, supporting PCIe 3.1 x4 and x8 interfaces with a maximum theoretical bandwidth of around 3.94 GB/s and 7.88 GB/s in both directions. Eventually, the modules could gain an x16 interface featuring 8 GT/s, 16 GT/s (PCIe Gen 4) or even 25 - 32 GT/s (PCIe Gen 5) data transfer rate (should the industry need SSDs with ~50 - 63 GB/s throughput). In fact, connectors are ready for PCIe Gen 5 speeds even now, but there are no hosts to support the interface.

EDIT: According to SFF-TA-1007 maximum interface for long ruler is x8:

 

[cbn]

If the top die for 3rd Gen IMFT is indeed 96L 16nm 2048Gb 3D QLC (1536Gb 3D TLC) that means per GB parallelism would be lower than the 64L 20nm 1024Gb 3D QLC.

With the current 32TB Long ruler form factor SSD likely have two NAND dies per chip enabler for a write speed of 1800 MB/s.....does a 64TB Long ruler use PCIe 3.0 x8* and one 2048Gb die per chip enable in order to regain write speed per TB?

*Thinking this would be two of the Intel designed 18 channel PCIe 3.0 x4 NAND controllers combined together as one 36 channel PCIe 3.0 x 8 controller. (This would match perfectly the 36 media spots available on the Long Ruler form factor)

 

[Billy Tallis]

Thinking this would be two of the Intel designed 18 channel PCIe 3.0 x4 NAND controllers combined together as one 36 channel PCIe 3.0 x 8 controller. (This would match perfectly the 36 media spots available on the Long Ruler form factor)

Intel's second generation NVMe controllers (P4xxx models) dropped down to 12 channels, and compensated with higher bus speeds. NAND bus speeds are still on the rise, though I don't recall off the top of my head how far Intel is behind Samsung on that score.

 

[cbn]

Intel's second generation NVMe controllers (P4xxx models) dropped down to 12 channels, and compensated with higher bus speeds. NAND bus speeds are still on the rise, though I don't recall off the top of my head how far Intel is behind Samsung on that score.

Thank you for the information.

Do you happen to know how many chip enablers the new 12 channel controller has?

EDIT: Tom's is reporting 4 chip enablers per channel (which would work out to be 48 chip enablers*), but then they are also mentioning the new controller is more scalable than previous versions---> https://www.tomshardware.com/news/intel-ssd-p4550-p4600-dc,34301.html

*48 chip enablers would only allow for a 6TB SSD using 64L 1024Gb 3D QLC (if using one die per chip enabler)

Intel's vertically integrated design employs the company's own NAND, controller, firmware, and components. The new fourth-generation controller features 12 channels (four CE per channel), whereas previous-generation controllers employed 18 channels. Paring back the channel count confers reduced power consumption, but in this case, it also yields a net performance gain. The performance increase is surprising in light of the transition to TLC NAND, which typically results in lower performance. Intel employs a dual-PCB design to house the hefty allotment of NAND, but it hasn't specified which capacity points leverage a daughterboard. The company also claims the new controller is more scalable than previous generations, so we could see higher capacities in the future with the same platform.

I wonder what they mean by scalable?

More chip enablers? (Normally I see eight chip enablers per channel on the higher end SSDs)

More channels? (Could the 12 channel controller be in reality a native 16 or 18 channel design with some channels disabled....and these channels can be re-enabled?)

Or does this merely refer to a higher Max addressable capacity with a static 12 channels and 4 chip enablers per channel.....and that Intel has another controller based on a larger die handling higher capacity needs?