I think this needs to end, Nvidia should to stop selling flagship cards and make them available through subscription only. $200/month and you get your 6090 delivery, no more bids and wasted time. For a premium $300/month you can even get early access to new cards before the press does (24 months subscription only, terms and conditions apply)
Subscription GPUs or any hardware sub will never take off. It’s not a good business model.
You would think so, but the car companies have all gone in this direction years ago. Tesla/BMW/Merc all charge annual fees for features. Same with consoles. I could see it happening on PCs and phones at some point.Subscription GPUs or any hardware sub will never take off. It’s not a good business model.
Yeah, but thats software locked features. The hardware is built into the cars already.This is what it takes to get one. I will hang on to my 4090, got some good resale offers on it but I don't want to do this.
You would think so, but the car companies have all gone in this direction years ago. Tesla/BMW/Merc all charge annual fees for features. Same with consoles. I could see it happening on PCs and phones at some point.
Apparently, DeepSeek R1 requires only 1/25th to 1/30th of the resources. Thats like just less than 4% (or even roughly 3%) of the resources compared to older (and now possibly obsolete) models like ChatGPT. DeepSeek makes ChatGPT look like dinosaurs that just entered extinction.Can someone ELI5 why DeepSeek being succesful would crush Nvidia prices? Is it not running on Nvidia GPUs?
Those videos are likely not running DeepSeek R1, but rather a hybrid LLM that distills the reasoning model of R1 into a smaller Llama or Qwen model. (that fits into the Arc 580 VRAM)There are videos on YouTube that show DeepSeek running on a Intel Arc 580 at usable speeds. Imagine that!
Using the reasoning data generated by DeepSeek-R1, we fine-tuned several dense models that are widely used in the research community. The evaluation results demonstrate that the distilled smaller dense models perform exceptionally well on benchmarks. We open-source distilled 1.5B, 7B, 8B, 14B, 32B, and 70B checkpoints based on Qwen2.5 and Llama3 series to the community.
DeepSeek-R1-Distill models are fine-tuned based on open-source models, using samples generated by DeepSeek-R1.We slightly change their configs and tokenizers. Please use our setting to run these models.
That's not the problem for Nvidia, the problem is (if it's true) training that model only cost 6 mln USDThere are videos on YouTube that show DeepSeek running on a Intel Arc 580 at usable speeds.
Apparently, DeepSeek R1 requires only 1/25th to 1/30th of the resources. Thats like just less than 4% (or even roughly 3%) of the resources compared to older (and now possibly obsolete) models like ChatGPT. DeepSeek makes ChatGPT look like dinosaurs that just entered extinction.
There are videos on YouTube that show DeepSeek running on a Intel Arc 580 at usable speeds. Imagine that! If we patiently sift thru all the noise, one thing is clear. What just happened is a revolution in AI. How big of an revolution... that we can see in Nvidia stock prices.
Is there a nice article that shows the performance dependence of the number of parameters in the model? Like what's the smallest reasonable model?Full DeepSeek V3 is 671 Billion Parameters, and needs over 1500 GB of VRAM... That's not running on any desktop GPU.
You can make small cut down models just like you can for other models.
There is a cut down DeepSeek 7 Billion parameter LLM model that will fit in 16GB, just as there are other cut down models...
Nvidia is screwed. They're not walking out of this one with flying colors. Imagine all the people who spent billions building AI data centers. There's no easy way for them to make back their money. If it's Microsoft, Facebook, Google, Elon Musk, etc, I think it's kinda okay as they can take the hit. They rest, I pray for them.That's not the problem for Nvidia, the problem is (if it's true) training that model only cost 6 mln USD
I still really don't get it... but maybe you guys are smarter than me.Nvidia is screwed. They're not walking out of this one with flying colors. Imagine all the people who spent billions building AI data centers. There's no easy way for them to make back their money. If it's Microsoft, Facebook, Google, Elon Musk, etc, I think it's kinda okay as they can take the hit. They rest, I pray for them.
Nvidia is now in such a precarious position, there's no simple way out. Looks like the AI bubble has finally burst!
DeepSeek enables smaller but very capable models to run on very few GPUs. Someone on X/Twitter said he now runs his own instance locally in his 4090 that does code gen. It's nothing short of a miracle for a developer. No internet connection needed. No subscription needed. And no query limits. Like it's totally free. And as capable as o1.I still really don't get it... but maybe you guys are smarter than me.
Like, if somebody invents a great new AI model, how is that going to be bad for the leader in GPUs?
Sure there's an order of magnitude efficiency gain in the training... won't they just make even bigger models then that are even better?
I don't get the problem for Nvidia. For OpenAI and Meta, I understand since this is an open model.
I think it's more just the collective shock of how much a unicorn tech bro bubble this was, that you don't have to pour 500B in to get meaningful results and the whole AI hype machine was built just as a pump and dump... maybe that I can see. But it feels like virtually anything could have triggered that realization.
Anyway, I'm happy there is a new and competitive open model, even if it wrecks my 401k.
If it is "good enough" then many small companies can sell AI as a commodity or or let it be part of the usual "you are the product payment" and in that case the demand for AI will still grow really fast, but the demand for chips will grow much slower, because of you can get good enough for "free" only very few will pay for really expensive models.I still really don't get it... but maybe you guys are smarter than me.
Like, if somebody invents a great new AI model, how is that going to be bad for the leader in GPUs?
Sure there's an order of magnitude efficiency gain in the training... won't they just make even bigger models then that are even better?
I don't get the problem for Nvidia. For OpenAI and Meta, I understand since this is an open model.
I think it's more just the collective shock of how much a unicorn tech bro bubble this was, that you don't have to pour 500B in to get meaningful results and the whole AI hype machine was built just as a pump and dump... maybe that I can see. But it feels like virtually anything could have triggered that realization.
Anyway, I'm happy there is a new and competitive open model, even if it wrecks my 401k.
New 6090 offers 70% more performance for a massive value of $999.99 🤣DeepSeek enables smaller but very capable models to run on very few GPUs. Someone on X/Twitter said he now runs his own instance locally in his 4090 that does code gen. It's nothing short of a miracle for a developer. No internet connection needed. No subscription needed. And no query limits. Like it's totally free. And as capable as o1.
In essence, when the requirements for AI come down drastically (both training & users), it directly hurts Nvidia sales as they're the predominant supplier. Of course, GPUs will always be needed for stuff like AGI. But for most common use cases, DeepSeek has literally made it free for many which kinda breaks Nvidia's backbone.
Nvidia's Revenue/profit projections were made on increasing sales of very expensive products, with demand growing quickly into the future. If you now need a much smaller amount of GPUs to deliver the same output, then all of those projections are worthless (worth less) by a lot.I still really don't get it... but maybe you guys are smarter than me.
Like, if somebody invents a great new AI model, how is that going to be bad for the leader in GPUs?
Sure there's an order of magnitude efficiency gain in the training... won't they just make even bigger models then that are even better?
I don't get the problem for Nvidia. For OpenAI and Meta, I understand since this is an open model.
I think it's more just the collective shock of how much a unicorn tech bro bubble this was, that you don't have to pour 500B in to get meaningful results and the whole AI hype machine was built just as a pump and dump... maybe that I can see. But it feels like virtually anything could have triggered that realization.
Anyway, I'm happy there is a new and competitive open model, even if it wrecks my 401k.
How much are you willing to bet on the bigger models? A few coins, a month's salary? There's a degree of hesitation in the market right now, investors had a wake-up call that was long overdue. Obviously nothing is certain and DeepSeek is riding a hype wave, but so were OpenAi until recently... it was all hype (in terms of investment confidence). If OpenAI can generate FOMO using smoke and mirrors, turns out another player can generate FUD using the same tricks.Like, if somebody invents a great new AI model, how is that going to be bad for the leader in GPUs?
Sure there's an order of magnitude efficiency gain in the training... won't they just make even bigger models then that are even better?
Patty G disagrees wholeheartedly.Nvidia is screwed. They're not walking out of this one with flying colors. Imagine all the people who spent billions building AI data centers. There's no easy way for them to make back their money. If it's Microsoft, Facebook, Google, Elon Musk, etc, I think it's kinda okay as they can take the hit. They rest, I pray for them.
Nvidia is now in such a precarious position, there's no simple way out. Looks like the AI bubble has finally burst!
Is there a nice article that shows the performance dependence of the number of parameters in the model? Like what's the smallest reasonable model?
Nothing to do with human labour. Its entire due to the rumour that DeepSeek is a side project from a Chinese quant company. Lots of very very smart math heavy folk in the company with access to some nice gpu's that are idle during night time. The company is already profitable from their quant trading and so don't need a business model for their AI models. The reported number of gpu hours is so low that if you were to rent from a cloud provider to train the latest DeepSeek R1 it would cost $5.5mil. But this is a rumour. Could just be a brag.I think there is a lot of exaggeration out there. From what I've read they are just combining already known efficiency techniques. It probably requires a lot more human labor to set it up, but being in China where human labor is cheaper and AI HW restricted, it's a perfect fit. In the west they may keep doing more brute force.
Commonly models are not run with the weights in BF16 format, but rather in 8 bits or lower. The weights are quantised. So 7b model can run in 7GB of vram + overhead + context. Or at q4 for 3.5GB vram but not so good quality. It depends on what you need.There is a cut down DeepSeek 7 Billion parameter LLM model that will fit in 16GB, just as there are other cut down models...
This is nothing new. Folk have been running AI models locally for a long while. To be able to run the full DeepSeek R1 at home, you would need a serious amount of hardware and then be happy with a serious power bill. Most likely they are running a smaller distilled version which doesn't have R1's full quality but is worth running. But software developers have been using AI models for a while to help them scale their own personal productivity.DeepSeek enables smaller but very capable models to run on very few GPUs. Someone on X/Twitter said he now runs his own instance locally in his 4090 that does code gen. It's nothing short of a miracle for a developer. No internet connection needed. No subscription needed. And no query limits. Like it's totally free. And as capable as o1.
I think we can all agree the news is bad for Nvidia the meme stock in the short term here. I think it remains to be seen if it’s bad for their fundamentals (revenue) in the longer term.How much are you willing to bet on the bigger models? A few coins, a month's salary? There's a degree of hesitation in the market right now, investors had a wake-up call that was long overdue. Obviously nothing is certain and DeepSeek is riding a hype wave, but so were OpenAi until recently... it was all hype (in terms of investment confidence). If OpenAI can generate FOMO using smoke and mirrors, turns out another player can generate FUD using the same tricks.
This video from Asianometry does a good job to present the recent AI landscape, with both a healthy dose of skepticism and a good dose of optimism:
It has an even smaller moat there I think. Unless they bought patents recently.If the AI bubble pops, NV will pivot to robotics..
They got on early in the crypto train. And so with AI. I'm sure they have a crack team working on the next big thing already. Thats what they do.It has an even smaller moat there I think. Unless they bought patents recently.
This is very hard to do on a consistent basis. But if they keep up the R/D I'm sure they will have a rosy future, although maybe not valued at $3T.Thats what they do.
I think this needs to end, Nvidia should to stop selling flagship cards and make them available through subscription only. $200/month and you get your 6090 delivery, no more bids and wasted time. For a premium $300/month you can even get early access to new cards before the press does (24 months subscription only, terms and conditions apply)
Meanwhile everyone else can buy prepaid cards, with credit on them. Buy a $300 5060 with 300 hours of gaming on it and 6 months credit life. The more you buy, the more you save!