beginner99
Diamond Member
- Jun 2, 2009
- 5,305
- 1,738
- 136
That price is insane. We got an A100 couple years back for less than 10k. But demand now is of course much, much higher. But as soon as you can get the same inference performance from a smaller, less power hungry chip that probbaly costs 1/3 to produce, that market will erode quickly. Most people don't need to train a LLM. But NV themselves are likley working on such products.Did you see H100 GPUs (AI chips really) are selling for $40k each? It might be a rough off ramp. I assuming they are planning for that somehow…
The hype may die down, but deep Learning usage is only going to get more pervasive.
Data Centers likely also have use cases for GPU compute, so they can buy Data Center GPUs that do both.
Dedicated HW will eat some market share, but there is nothing stopping NVidia from also making dedicated Tensor parts for Data Center if there was a strong move away from GPUs. Dedicated HW actually came first as Google was making their own dedicated Tensor units before NVidia added the functionality.
I agree, usage is going up because big corporations adaptiing takes time but then in 1-3 years when al the big gains promised are not really measurable at all except the fortune you paid for the AI model, then things will quickly cool down. And yeah pretty sure NV is also doing R&D for simple inferencing cards.