Even at a sample size of 30+M, is the Steam GPU survey accurate?

Page 3 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

notty22

Diamond Member
Jan 1, 2010
3,375
0
0
A benchmark and a poll are apples and oranges. A benchmark is measuring a reproducible result based on the performance of the part. A poll is taking a sample of what is out there and is subject to an error variance.

Take for instance presidential polls. Those have error ranges upto 5-7%. We are talking a possible .5-1.0% error rage here. That is miniscule. People's GPU's that dont show up in the survey or represents too small of a sample to show as an independent data point fall under the title of "other". "Other" represents .74% of DX11 GPU's as of April.


Agreed, I had a post typed about addressing the 'other' column of the Steam survey.

IMO, as part of the EULA agreement we sign with Steam, they probably do a scan of your hardware and get the exact gpu your are using from the card/vendor ID. Nvidia and AMD or any developer know exactly how to read that information. As far as being asked questions, those are probably other aspects of the same or different survey.
 

Lonyo

Lifer
Aug 10, 2002
21,939
6
81
A benchmark and a poll are apples and oranges. A benchmark is measuring a reproducible result based on the performance of the part. A poll is taking a sample of what is out there and is subject to an error variance.

Take for instance presidential polls. Those have error ranges upto 5-7%. We are talking a possible .5-1.0% error rage here. That is miniscule. People's GPU's that dont show up in the survey or represents too small of a sample to show as an independent data point fall under the title of "other". "Other" represents .74% of DX11 GPU's as of April.

You're assuming the complete dropoff of one card is representative of the error margin of the survey.
It's not. I cited it as an example of the unreliability of what we see. The error margin is much larger when looking at Steam users vs the overall population.

So basically the Steam survey has a margin of error of at least 0.54% we can assume, meaning that anything with low percentages is unreliable.

We also know that Steam doesn't represent overall sales, because it has a ridiculous margin of error when compared to relative sales numbers in cards which all cost >$100.

We also know it simply fails to be even remotely accurate in some situations, e.g. Crossfire detection vs SLI.

Some cards simply don't show up at times even when they were released concurrently with other cards for some reason.

So we can't use it to compare cards with small % representation.
We can't use it to give overall market stats at a $100 price floor.
We can't use it to demonstrate differences between multi GPU adoption.
We can't use it to reflect sales of brand new cards.

So what it it useful for in any meaningful statistical way?
Maybe looking at broad patterns of adoption of new technology (e.g. DX11 systems) by gamers who use Steam, and a broad idea of the general level of hardware which the majority have, but it doesn't even give an accurate month to month snapshot because of the random nature of the selection and the selection bias of Steam users, and it doesn't accurately reflect sales between NV and AMD or within NV and AMD product lines.

It is not ACCURATE at all, it can be used to gain a broad picture of the overall direction the market is taking and the overall potential level of hardware that might be broadly found in end user computers, but it isn't ACCURATE to any degree, especially not in the way discussed by the OP, and nor in the way suggested by most of the Steam hardware survey threads that pop up.
 

Genx87

Lifer
Apr 8, 2002
41,095
513
126
You're assuming the complete dropoff of one card is representative of the error margin of the survey.
It's not. I cited it as an example of the unreliability of what we see. The error margin is much larger when looking at Steam users vs the overall population.

And what do you attribute it to?

So basically the Steam survey has a margin of error of at least 0.54% we can assume, meaning that anything with low percentages is unreliable.

Possibly which is why I said watch the trend over the course of time. If it doesnt vary much, it is pretty reliable.

We also know that Steam doesn't represent overall sales, because it has a ridiculous margin of error when compared to relative sales numbers in cards which all cost >$100.

You consider .5% a ridiculous margin of error?????

We also know it simply fails to be even remotely accurate in some situations, e.g. Crossfire detection vs SLI.

Ok? Do you feel crossfire and SLI represent enough of the market it would really impact the overall information?

Some cards simply don't show up at times even when they were released concurrently with other cards for some reason.

So what are you saying? They fall under the "other" category, or are being lopped onto sales of the other card? 6850 was being counted as a 6870?

So we can't use it to compare cards with small % representation.
We can't use it to give overall market stats at a $100 price floor.
We can't use it to demonstrate differences between multi GPU adoption.
We can't use it to reflect sales of brand new cards.

Sure we can by watching the trends and understanding there will be some error in the process as it is a "survey".

So what it it useful for in any meaningful statistical way?
Maybe looking at broad patterns of adoption of new technology (e.g. DX11 systems) by gamers who use Steam, and a broad idea of the general level of hardware which the majority have, but it doesn't even give an accurate month to month snapshot because of the random nature of the selection and the selection bias of Steam users, and it doesn't accurately reflect sales between NV and AMD or within NV and AMD product lines.

I am not sure what you are expecting out of this survey. There is going to be a margin of error. So far we seem to agree it is right between .5-1.0% points. That is miniscule in a survey and damn accurate if true. Much more accurate than public opinion surveys.

It is not ACCURATE at all, it can be used to gain a broad picture of the overall direction the market is taking and the overall potential level of hardware that might be broadly found in end user computers, but it isn't ACCURATE to any degree, especially not in the way discussed by the OP, and nor in the way suggested by most of the Steam hardware survey threads that pop up.

What do you consider an accurate picture of the landscape then? Most marketshare numbers have Nvidia holding right around 60% marketshare in desktop discrete graphics. Look at the 570 and 580 vs the 6950 and 6970. Coincidence they(570+580) are showing up on Steam at a 2:1 clip compared to their competition?
 

Lonyo

Lifer
Aug 10, 2002
21,939
6
81
The margin of error overall is shown earlier in the thread in a post I made, where there's a real sales numbers split of 37.5/62.5 between the 5800 and 5700 series, and a 56/44 split according to Steam.

That's for "overall market" usage.
And 0.56% on an overall market is more like 2% on a DX11 market (since the DX11 market is around 1/4th of the overall market). And if you have something on 2.5 and something else on 1.85, then 2% means it could be .5% vs 3.85%, more than a complete reversal of what the initial numbers show, so 2% is fucking HUGE in that context.
 

OCGuy

Lifer
Jul 12, 2000
27,227
36
91
Simple, the 5XXX series dominated DX11.

-They had DX11 all to themselves for seven months.

-The 5770 is the best selling DX11 card by a wide margin.

-The 5850 is still today the best value going in a DX11 card.

-They likely unloaded piles of 5870s when there was no competition for those cards. A high margin card.

-They've been scooping up mobile gpu market share.

-They've had the fastest card on the market since Sept/09 until today and that helps to command sales.

Did you copy that off an AMD marketing slide or something? Yikes...


With all of that, the marketshare is 20% to 24%, I wouldnt call that "domination."
 

SHAQ

Senior member
Aug 5, 2002
738
0
76
I would think 30m+ would be a pretty damn good sample size. How many 360/PS3/Wii systems have been sold? Welcome to reality I guess. The sample size is becoming even more relevant as people don't need a PC for email, web surfing or casual gaming. Nvidia is more mod friendly and has had user editable game profiles for at least 5 years now. Their drivers are generally better as well especially for OpenGL. Most people don't worry about saving 20-30 watts per card. Change your light bulbs to high efficiency and don't worry about it.
 

tijag

Member
Apr 7, 2005
83
1
71
I would think 30m+ would be a pretty damn good sample size. How many 360/PS3/Wii systems have been sold? Welcome to reality I guess. The sample size is becoming even more relevant as people don't need a PC for email, web surfing or casual gaming. Nvidia is more mod friendly and has had user editable game profiles for at least 5 years now. Their drivers are generally better as well especially for OpenGL. Most people don't worry about saving 20-30 watts per card. Change your light bulbs to high efficiency and don't worry about it.

Wow. So many things wrong with this post.

First of all, WAAAAY off topic. Just your personal thumbs up on Nvidia cards and drivers. We don't need random chime ins with opinion on who makes the best drivers, in a thread about the relevance of the Steam hardware survey.

Secondly, as has been pointed out, they aren't sampling 30 million users. Its a survey sent out at random to a very small sample of their overall user base, and their detection software apparently has a hard time identifying the second GPU in CF and often can't identify other AMD GPU's.

Why did you make this post? Its as if you were having a conversation in real life with someone, and it accidentally spilled over into this thread.
 

Lonyo

Lifer
Aug 10, 2002
21,939
6
81
BRB, surveying the entire 37 million population of California so that I can say I have an accurate picture of the USA as a whole, since that's over 10% of the population so it must be representative. I mean, most surveys with fairly low margins of error have a lot fewer participants.
 

Jacky60

Golden Member
Jan 3, 2010
1,123
0
0
GTX 580 owners tend to be smarter, better looking, and drive nice cars.

6990 owners are especially handsome and attractive to the ladies but also modest about their abundant talents and ability to choose the very fastest kit out there.
 
Feb 19, 2009
10,457
10
76
If you look at the figures, it has the Mobility Radeon 5870 nearing the % of 6870 and 6850. That's just not right because gaming laptops (giant bricks with 30min battery life) just don't sell that well.

AMD has previously said they sold in the double digit millions of 6xxx and their quarterly reports show big gains. They are doing fine.
 

Puffnstuff

Lifer
Mar 9, 2005
16,038
4,800
136
You know I'm really close to buying one of those desktop replacement gaming bricks with a 6970m or two in it.
 

notty22

Diamond Member
Jan 1, 2010
3,375
0
0
TH noted in their gtx 560 review how in the latest Steam Survey the new primary desktop resolution is 1920x1080 ( I had noticed this to ).
I'm surprised how that has changed in the past year.
IMO, this has to lead to people needing more gpu power for gaming.
Which is a good trend, I think
 

DirkGently1

Senior member
Mar 31, 2011
904
0
0
TH noted in their gtx 560 review how in the latest Steam Survey the new primary desktop resolution is 1920x1080 ( I had noticed this to ).
I'm surprised how that has changed in the past year.
IMO, this has to lead to people needing more gpu power for gaming.
Which is a good trend, I think

21%, just a shade ahead of 1680x1050 which is 19%. It's just a shame that 16:9 is becoming standard ahead of 16:10.

FWIW, i think the survey is a nice feature and of great value.
 
sale-70-410-exam    | Exam-200-125-pdf    | we-sale-70-410-exam    | hot-sale-70-410-exam    | Latest-exam-700-603-Dumps    | Dumps-98-363-exams-date    | Certs-200-125-date    | Dumps-300-075-exams-date    | hot-sale-book-C8010-726-book    | Hot-Sale-200-310-Exam    | Exam-Description-200-310-dumps?    | hot-sale-book-200-125-book    | Latest-Updated-300-209-Exam    | Dumps-210-260-exams-date    | Download-200-125-Exam-PDF    | Exam-Description-300-101-dumps    | Certs-300-101-date    | Hot-Sale-300-075-Exam    | Latest-exam-200-125-Dumps    | Exam-Description-200-125-dumps    | Latest-Updated-300-075-Exam    | hot-sale-book-210-260-book    | Dumps-200-901-exams-date    | Certs-200-901-date    | Latest-exam-1Z0-062-Dumps    | Hot-Sale-1Z0-062-Exam    | Certs-CSSLP-date    | 100%-Pass-70-383-Exams    | Latest-JN0-360-real-exam-questions    | 100%-Pass-4A0-100-Real-Exam-Questions    | Dumps-300-135-exams-date    | Passed-200-105-Tech-Exams    | Latest-Updated-200-310-Exam    | Download-300-070-Exam-PDF    | Hot-Sale-JN0-360-Exam    | 100%-Pass-JN0-360-Exams    | 100%-Pass-JN0-360-Real-Exam-Questions    | Dumps-JN0-360-exams-date    | Exam-Description-1Z0-876-dumps    | Latest-exam-1Z0-876-Dumps    | Dumps-HPE0-Y53-exams-date    | 2017-Latest-HPE0-Y53-Exam    | 100%-Pass-HPE0-Y53-Real-Exam-Questions    | Pass-4A0-100-Exam    | Latest-4A0-100-Questions    | Dumps-98-365-exams-date    | 2017-Latest-98-365-Exam    | 100%-Pass-VCS-254-Exams    | 2017-Latest-VCS-273-Exam    | Dumps-200-355-exams-date    | 2017-Latest-300-320-Exam    | Pass-300-101-Exam    | 100%-Pass-300-115-Exams    |
http://www.portvapes.co.uk/    | http://www.portvapes.co.uk/    |