Discussion Apple Silicon SoC thread

[Eug]

M1
5 nm
Unified memory architecture - LP-DDR4
16 billion transistors

8-core CPU

4 high-performance cores
192 KB instruction cache
128 KB data cache
Shared 12 MB L2 cache

4 high-efficiency cores
128 KB instruction cache
64 KB data cache
Shared 4 MB L2 cache
(Apple claims the 4 high-effiency cores alone perform like a dual-core Intel MacBook Air)

8-core iGPU (but there is a 7-core variant, likely with one inactive core)
128 execution units
Up to 24576 concurrent threads
2.6 Teraflops
82 Gigatexels/s
41 gigapixels/s

16-core neural engine
Secure Enclave
USB 4

Products:
$999 ($899 edu) 13" MacBook Air (fanless) - 18 hour video playback battery life
$699 Mac mini (with fan)
$1299 ($1199 edu) 13" MacBook Pro (with fan) - 20 hour video playback battery life

Memory options 8 GB and 16 GB. No 32 GB option (unless you go Intel).

It should be noted that the M1 chip in these three Macs is the same (aside from GPU core number). Basically, Apple is taking the same approach which these chips as they do the iPhones and iPads. Just one SKU (excluding the X variants), which is the same across all iDevices (aside from maybe slight clock speed differences occasionally).

EDIT:



M1 Pro 8-core CPU (6+2), 14-core GPU
M1 Pro 10-core CPU (8+2), 14-core GPU
M1 Pro 10-core CPU (8+2), 16-core GPU
M1 Max 10-core CPU (8+2), 24-core GPU
M1 Max 10-core CPU (8+2), 32-core GPU

M1 Pro and M1 Max discussion here:

Page 78 - Discussion - Apple Silicon SoC thread

Page 78 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

M1 Ultra discussion here:

Page 109 - Discussion - Apple Silicon SoC thread

Page 109 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

M2 discussion here:

Page 127 - Discussion - Apple Silicon SoC thread

Page 127 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

Second Generation 5 nm
Unified memory architecture - LPDDR5, up to 24 GB and 100 GB/s
20 billion transistors

8-core CPU

4 high-performance cores
192 KB instruction cache
128 KB data cache
Shared 16 MB L2 cache

4 high-efficiency cores
128 KB instruction cache
64 KB data cache
Shared 4 MB L2 cache

10-core iGPU (but there is an 8-core variant)
3.6 Teraflops

16-core neural engine
Secure Enclave
USB 4

Hardware acceleration for 8K h.264, h.264, ProRes

M3 Family discussion here:

Page 215 - Discussion - Apple Silicon SoC thread

Page 215 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

M4 Family discussion here:

Page 263 - Discussion - Apple Silicon SoC thread

Page 263 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

forums.anandtech.com

 

[poke01]

This a thread about Apples GPUs found in M3 and later.

https://twitter.com/x/status/1806244885684490526

 

[okoroezenwa]

This a thread about Apples GPUs found in M3 and later.

https://twitter.com/x/status/1806244885684490526

Interesting. I knew he was pretty impressed last year when the M3s were released but wasn’t sure with his recent threading. Looks like he still feels that way about Apple’s GPUs.

Related, what do we think about the configuration of the A18? I’m wondering since we have the M4 out, with CPU changing meaningfully but GPU seems to just be higher clocked M3/A17 GPU cores as far as I’m aware. Will the A18 be a smaller M4 or are GPU (and even CPU) enhancements over what we’ve seen in M4 expected? Also ANE if anyone cares.

 

[FlameTail]

If A18 is going to have a new GPU/NPU, then will those advancements also come to the M4 Pro/Max?

Would be bizarre if it didn't.

 

[TwistedAndy]

I’m specifically talking about inference for a reason, and it is because Macs can inference with large models at a decent pace and Thunderbolt is proving sufficient in some cases for linear scaling, so not so much of a bottleneck.

Apple Silicon has the advantage of a bigger memory size, but the performance is pretty low. Here is a comparison in llama.cpp.

If we take 70B Q4 model, Apple M2 Ultra generates 4.76 t/s in the text generation mode and 147.58 t/s in prompt processing. M3 Max - 7.65 t/s and 70.19 t/s. Old nVidia RTX A6000 48GB ($4K on Amazon) achieves a few times better performance: 14.71 t/s and 482.19 t/s.

I think Apple will introduce smaller 3B/8B models with aggressive quantization, which can work on 16GB Apple Silicon devices.

 

[Doug S]

They can't release new stuff when they have tons of unsold chips and possibly millions of base model laptops in their warehouses. Releasing M4 will make it harder for them to offload their inventory of M3 chips (and maybe even M2 too!). The only way they can avoid this is if they made some sort of deal with TSMC where they can signal to TSMC to immediately stop production and move to producing new chips, regardless of the impact that has on TSMC's bottom line because obviously stopping M3 chip production would mean nothing owed to TSMC for chips not made and then TSMC is on their own to find someone else to fill the fabs that were previously making the M3 chips. If Apple has a deal to make 50 million M3 chips, that's what they get or they have to pay some early termination fee to stop production and move to newer chip production on better process.

Sorry try again with your stupid theories about mountains of unsold inventory:

https://9to5mac.com/2024/06/26/us-mac-market-share-q1-2024/

 

[igor_kavinski]

Sorry try again with your stupid theories about mountains of unsold inventory:

https://9to5mac.com/2024/06/26/us-mac-market-share-q1-2024/

Apple Mac shipments worldwide by quarter 2023 | Statista

In the fourth quarter of 2023, Apple Mac shipments stood at 6.3 million units worldwide, an increase from the 5.9 million Mac shipments that were recorded in the fourth quarter of the previous year.

www.statista.com

The demand peaks are fine but it's the sales slump nosedives that might cause inventory to build up. Obviously I don't have any real data but if I were an Apple investor, I would be interested in Apple's official strategy regarding unsold inventory.

 

[The Hardcard]

Unfortunately, things are much more complicated. Apple Silicon devices with a sufficient amount of memory can run relatively big inference models (70-180B), but the performance will not be so great. It's much more efficient to use smaller and much faster 3-7B models.

If we consider training and other more complex ML tasks, nVidia cards are much better in terms of software, performance, and scalability. There are reasons why nVidia develops NVLink, dedicated networking hardware, and systems like NVL72. For example, the latest NVLink version offers 1,800 GB/s GPU-to-GPU bandwidth. Thunderbolt 4 offers only 4 GB/s.

In terms of software, nVidia has been developing CUDA for 17 years. There is a huge amount of documentation, examples, libraries, integrations, developer programs, etc. And a lot of stuff is available for free and can run on regular nVidia cards. AMD and Intel are trying to implement their own CUDA alternatives (ROCm and openAPI), but they are far behind nVidia in many aspects.

From a company perspective, nVidia is a much safer choice. Yes, NVL72 costs a few millions of dollars, but it's an investment. AMD and Intel platforms look worse in general, but in some cases, they can fit well.

Apple is not an option here at all. It offers an even more restricted ecosystem than nVidia, but without its benefits.

I’m not talking about what Apple is doing for on device AI, nor even what models they are using in their private cloud, though it could be applicable. This is about people buying hardware to run high parameter models locally.

4-bit quantization has above 96 percent accuracy and larger parameter models even with that low precision are superior to smaller parameter models at full precision.

The M2 Ultra can generate 22.7 tokens per second 70B at 4 bits, above 15 words per second average, well within reasonable reading speed. Also one under $6K Studio can load models that would require $16K of those Nvidia (likely at least $20K for the upcoming Ultra box) in addition to many $Ks extra for one or more systems that can link 5 cards. The Nvidia solution is far faster for everyone with the extra money, space, and electrical capacity.

What I’m hoping is that Apple can address the compute limitations for incoming Macs. They already have a dead simple way bump bandwidth by up to 40 percent just by using faster memory modules, which will raise token generation.

But there are also several reasonable ways to significantly boost matmul performance to clean up some weaknesses. If done, Ultras can absolutely compete with Nvidia in a number of AI scenarios. The Ultra high memory capacity is meaningful to many and wil be moreso at 256 GB. So far there is no sign of any GPU cards below $10K adding VRAM.

 

[The Hardcard]

Apple Mac shipments worldwide by quarter 2023 | Statista

In the fourth quarter of 2023, Apple Mac shipments stood at 6.3 million units worldwide, an increase from the 5.9 million Mac shipments that were recorded in the fourth quarter of the previous year.

www.statista.com

View attachment 101997

The demand peaks are fine but it's the sales slump nosedives that might cause inventory to build up. Obviously I don't have any real data but if I were an Apple investor, I would be interested in Apple's official strategy regarding unsold inventory.

If it were inventory, putting the M2 or M3 in iPad Pros would be the move. I think the completely unexpected M4 release is strong evidence of a lack of inventory issues.

 

[igor_kavinski]

I think the completely unexpected M4 release is strong evidence of a lack of inventory issues.

Or they got really scared that Zen 5 would steal their thunder so they acted immediately as a pre-emptive strike for public mindshare. They more or less succeeded.

 

[okoroezenwa]

Or they got really scared that Zen 5 would steal their thunder so they acted immediately as a pre-emptive strike for public mindshare. They more or less succeeded.

Let’s be serious please 😂

Out of all of them AMD is probably the one that would cause Apple to move the least.

 

[The Hardcard]

Or they got really scared that Zen 5 would steal their thunder so they acted immediately as a pre-emptive strike for public mindshare. They more or less succeeded.

If that were the case, it doesn’t explain the long wait for M4 series Macs - the point of this line of discussion I thought - since the existence of the M4 is costing them M3 series Mac sales. Any possible inventory oversupply would be much worse now.

 

[defferoo]

Or they got really scared that Zen 5 would steal their thunder so they acted immediately as a pre-emptive strike for public mindshare. They more or less succeeded.

Please, you don't just pull in the launch of a SoC on a new process node in a new device on a whim. the amount of lead time needed to release the M4 iPad Pro is likely measured in years, not something you can just do. Apple probably does not care about Zen 5 much at all, I've never seen Apple compare one of their chips to an AMD chip, they only compare against their older products.

 

[igor_kavinski]

If that were the case, it doesn’t explain the long wait for M4 series Macs

You gotta understand human nature. If they release M4 Macs now, their M3 sales go down which means a loss on chips that they have already paid for and that are in their possession right now. Keeping M4 away from Macs means current owners of every Macbook from M1 to M3 may wanna get an iPad Pro. Not because they need it but because they want the fastest iPad ever that's faster than even their expensive Macbook. It's a pure marketing move to drive sales of a product that otherwise people would ignore, especially those that prefer laptops. But now, those same people WITH a Macbook are thinking to themselves, hey, my Macbook is getting a bit long in the tooth. Maybe I could use some new snappiness or I could multitask and offload some of my processing on iPad to get work done faster. An idle mind is a devil's workshop and who would be more devilish than someone with a pocket full of cash and itching to burn it on something? That's Apple's prime demographic. Most of these M4 sales are gonna be these people. The ones who really need it for genuine purposes are going to be a small percentage. Apple is in the business of fulfilling wants, not necessarily needs.

 

[igor_kavinski]

I've never seen Apple compare one of their chips to an AMD chip

But reviewers do.

 

[TwistedAndy]

The M2 Ultra can generate 22.7 tokens per second 70B at 4 bits, above 15 words per second average, well within reasonable reading speed. Also one under $6K Studio can load models that would require $16K of those Nvidia (likely at least $20K for the upcoming Ultra box) in addition to many $Ks extra for one or more systems that can link 5 cards. The Nvidia solution is far faster for everyone with the extra money, space, and electrical capacity.

What I’m hoping is that Apple can address the compute limitations for incoming Macs. They already have a dead simple way bump bandwidth by up to 40 percent just by using faster memory modules, which will raise token generation.

But there are also several reasonable ways to significantly boost matmul performance to clean up some weaknesses. If done, Ultras can absolutely compete with Nvidia in a number of AI scenarios. The Ultra high memory capacity is meaningful to many and wil be moreso at 256 GB. So far there is no sign of any GPU cards below $10K adding VRAM.

A MacBook or Mac Studio may be a good option if you want to do inference on really big LLM models.

If we consider commercial AI/ML projects, Apple Silicon is not an option in most cases. Having good hardware is not sufficient for success.

 

[The Hardcard]

You gotta understand human nature. If they release M4 Macs now, their M3 sales go down which means a loss on chips that they have already paid for and that are in their possession right now. Keeping M4 away from Macs means current owners of every Macbook from M1 to M3 may wanna get an iPad Pro. Not because they need it but because they want the fastest iPad ever that's faster than even their expensive Macbook. It's a pure marketing move to drive sales of a product that otherwise people would ignore, especially those that prefer laptops. But now, those same people WITH a Macbook are thinking to themselves, hey, my Macbook is getting a bit long in the tooth. Maybe I could use some new snappiness or I could multitask and offload some of my processing on iPad to get work done faster. An idle mind is a devil's workshop and who would be more devilish than someone with a pocket full of cash and itching to burn it on something? That's Apple's prime demographic. Most of these M4 sales are gonna be these people. The ones who really need it for genuine purposes are going to be a small percentage. Apple is in the business of fulfilling wants, not necessarily needs.

That post is a full-scale thermonuclear attack on Occam’s Razor.

 

[poke01]

Apple doesn’t care about new AMD CPUS, they always compared with Intels. Old or new.

In fact an AMD CPU was never even featured as a comparison point in an Apple keynote. AMD isn’t a some legendary company where Apple has to pay them attention.

 

[The Hardcard]

A MacBook or Mac Studio may be a good option if you want to do inference on really big LLM models.

Exactly! Precisely what I’m referring to. Because all the innovations boost the usefulness and appeal of low parameter models squeezed into less than 32 GB of RAM benefits high parameter models as well. Above 100 billion parameter models will be at the vanguard of neural network capabilities for the foreseeable future. Where Macs are already very competitive.If we consider commercial AI/ML projects, Apple Silicon is not an option in most cases. Having good hardware is not sufficient for success.

If we consider commercial AI/ML projects, Apple Silicon is not an option in most cases. Having good hardware is not sufficient for success.

No need for anywhere near most. If Apple decides to do things they are already technically capable of doing, a fraction of the market will result in the highest Mac sales ever.

 

[igor_kavinski]

AMD isn’t a some legendary company where Apple has to pay them attention.

Yup, a non-legendary company with legendary CPUs like Athlon64, Zen 3, 5800X3D, 5950X, 7800X3D, 7950X and Epyc Genoa.

The worst thing about being No.1 is the effort needed to stay No.1

And for a company like Apple with billions in its coffers, it would look embarrassing if some non-legendary company was suddenly heralded in the media as the maker of the fastest CPU in the world. Benchmarks matter. Apple ignoring AMD doesn't mean they don't perceive AMD as a threat. To do so would be foolish and unwise.

Apple doesn't matter to the average person. That's why they have to struggle harder to stay in the public mind.

 

[Doug S]

Yup, a non-legendary company with legendary CPUs like Athlon64, Zen 3, 5800X3D, 5950X, 7800X3D, 7950X and Epyc Genoa.

The worst thing about being No.1 is the effort needed to stay No.1

And for a company like Apple with billions in its coffers, it would look embarrassing if some non-legendary company was suddenly heralded in the media as the maker of the fastest CPU in the world. Benchmarks matter. Apple ignoring AMD doesn't mean they don't perceive AMD as a threat. To do so would be foolish and unwise.

Apple doesn't matter to the average person. That's why they have to struggle harder to stay in the public mind.

Why would that be embarrassing? It isn't as if Apple has had the fastest CPU every since M1 came out. They aren't even TRYING to have the fastest CPU, they care about having the fastest CPU when installed in the form factor of their devices, which (including iPad and very especially if you include iPhone) are passively cooled.

They don't care about stuff like Threadripper or Epyc, they aren't even trying to compete in that market and likely never will.

 

[FlameTail]

The Apple M4 release was pulled forward so that they could put it in the new iPad Pro. M4 has a new display engine to support the Tandem OLED of the iPad Pro, a stronger Neural Engine, etc...

 

[Eug]

The Apple M4 release was pulled forward so that they could put it in the new iPad Pro. M4 has a new display engine to support the Tandem OLED of the iPad Pro, a stronger Neural Engine, etc...

If it was pulled forward for the new display, then that pull forward was planned years ago. Way back in 2022, Ross Young told us that OLED was coming to the iPad Pro in 2024 H1.

 

[poke01]

Apple doesn't matter to the average person. That's why they have to struggle harder to stay in the public mind.

Look I get your biased towards AMD but in terms of Apple in the public mind Apple is ahead of AMD by a huge huge margin. Apple doesn’t have to struggle at all.

And for a company like Apple with billions in its coffers, it would look embarrassing if some non-legendary company was suddenly heralded in the media as the maker of the fastest CPU in the world. Benchmarks matter. Apple ignoring AMD doesn't mean they don't perceive AMD as a threat. To do so would be foolish and unwise.

The M4 is class leading in IPC and performance per watt and has industry leading GPU architecture.

Apple’s not even a chip company and yet designs better CPUs than Intel and AMD. Because Apple has billions they be self-reliant on the AI server space and use their own architecture instead of using AMD/Nvidia.

Yup, a non-legendary company with legendary CPUs like Athlon64, Zen 3, 5800X3D, 5950X, 7800X3D, 7950X and Epyc Genoa.

So? These are great chips but so is the A7, A9, A11, A14, A17 and M4.

The point is in their marketing Apple doesn’t mention AMD because they are irrelevant to their customers. This may change as AMD get more market share in laptops.

Apple ignoring AMD doesn't mean they don't perceive AMD as a threat. To do so would be foolish and unwise.

Internally in their testing labs they do test all kinds of CPUs, they have dedicated labs that test CPUs from different companies.

 

[The Hardcard]

The Apple M4 release was pulled forward so that they could put it in the new iPad Pro. M4 has a new display engine to support the Tandem OLED of the iPad Pro, a stronger Neural Engine, etc...

Was the M4 pulled forward? I’m not sure. I think it was late. I think they originally planned to sell products with this chip in 2022. People assess Apple’s plans like TSMC’s struggles, Covid and that year and a half supply chain crisis had no effect.

Not to mention the rumors of Apple’s design struggles, especially with the GPU. I don’t remember if I said it here or elsewhere, but I couldn’t help but notice how close the M3 Max GPU performance is to the marketing hype for the M1 Max. I don’t think it’s a coincidence.

 

[poke01]

@Eug the reason why Apple hasn’t launched the M4 in Macs yet is because of optics. They now want to release all the MBP’s at the same time that means the Max and Pro have to be ready.

That won’t happen till November, bigger chips require more time for validation etc.