Cheng Disputes AnandTech HQV Test Results

[geo1]

Here: http://www.hexus.net/content/item.php?item=3702&page=5

Addition: Godfrey mentions that ATI spent over three days trying to reproduce Anand's testing environment and results but with no joy, indicating that Anand's testing was flawed in some way. That backs up Godfrey's statement that the Intervideo decoder would have been the better one to use.

While it is hardly unknown for the "loser" to dispute the methods of the tester, has AT been in contact with Cheng to discuss this concern?

 

[imported_ST]

Funny... as Alan Dang who is commissioned to work on the HQV 2 software by Silicon Optix themselves, agrees with Anandtech's assessment:

"In that regard, it's somewhat ironic to read through Principled Technologies' commissioned study. They've made several errors in evaluating the video playback performance of PC GPUs resulting in 10 points of artificial inflated scores for NVIDIA and an artificial 5 point penalty for the i945G. In my testing, NVIDIA PureVideo solutions playing a DVD via a native 480p window using a 1280x1024 DVI connection resulted in only 51 out of 130 points. This is identical to the results obtained at Anandtech.com but clearly different from the 86 points that Hot Hardware gave or the 68 point that NVIDIA?s commissioned article gave."

see Firing Squad Video Quality Analysis report.

edit: unless Cheng is pointing to the ATI results, which Alan will look at later and AT admittedly says they did not use the Catalyst 5.9 drivers.

 

[Wreckage]

Well Cheng's title is "Director of Marketing", so he could never say "yep we suck".

PureVideo has almost a 2 year head start on AVIVO so it's not suprising it peforms better. I'm still waiting for a true H.264 player for either card, to see how well that works. Hopefully it will just be a codec plug-in for WMP.

 

[nRollo]

I was actually one of the people that was sent that software, I could run the benches and see if I replicate Anands results if anyone cares.

I didn't because I don't have a projector, so I'm not big on watching movies on the smallest screen in the house.

 

[rbV5]

Realistically, untill ATI produces the AVIVO solution from driver support to decoder support, they are at the mercy of the reviewers test bed. Obviously, you can get a Nvidia card, buy the purevideo decoder and play the media in WMP10 and have the supported Purevideo solution front to back. Can you do that with AVIVO...no, you can't.

Cheng should realize that ATI could simply produce their own front to back solution to compare AVIVO with Purevideo, but untill that happens, IMO using Purevideo decoders is OK with me, I use them with my ATI cards myself, and I'll have to say that IMO, the cyberlink decoders don't have an IQ edge on ATI hardware compared to Nvidia decoders in WMP10 or MCE 2005 (of course I have no x1000 card to test yet) but on legacy hardware, its not the case from my experience.

I don't recall seeing Cyberlink having any feature advantages with ATI hardware, like Purevideo decoders have with Nvidia hardware, so if you are going to "claim" them, "show" them, its not good enough just to throw it out there.

His responses about H.264 is not all that encouraging to me either. It sure "sounds" like H264 in an AVI container will be left out of AVIVO. That means alternative solutions like x264 are not going to realize any hardware decoding support if I "read" what he is saying correctly. Hopefully MP4 containers will be supported outright, but now I'm worried about my current encodings (good thing I've only encoded mostly SD media due to the poor software decoding of HD material)

 

[rbV5]

I didn't because I don't have a projector, so I'm not big on watching movies on the smallest screen in the house.

The tests are not "watching" movies, its watching selected scenes from the test media to test selected performance areas and to score your hardware based on those subjective tests.

I think the issue is not so much the Nvidia results anyway, but rather on the methods to test the ATI hardware.

 

[nRollo]

Originally posted by: rbV5

I didn't because I don't have a projector, so I'm not big on watching movies on the smallest screen in the house.

The tests are not "watching" movies, its watching selected scenes from the test media to test selected performance areas and to score your hardware based on those subjective tests.

I think the issue is not so much the Nvidia results anyway, but rather on the methods to test the ATI hardware.

I didn't mean I didn't want to watch the tests, I meant the video decode is a secondary issue for me because I don't use my PCs for home theatre like many do.

Point taken on AVIVO being a work in progress, that's not surprising at this point. (hasn't been around long)

 

[rbV5]

I didn't mean I didn't want to watch the tests, I meant the video decode is a secondary issue for me because I don't use my PCs for home theatre like many do.

You can use it to test your theater equipment or DVD players ect, its not just for PC's. Of course, if you have no use for the DVD.......

 

[geo1]

If I understand Cheng's comments correctly in the Hexus interview, he's not disputing the results in HQV for PureVideo. He works his marketing jujitsu on that in another way, claiming (in so many words) that PureVideo is a one-trick pony, and that Standard Def DVD (which is what HQV tests) is all PureVideo does well.

No, his main gripe seems to be that the AT piece did not do justice on the Avivo results side, with the Intervideo decoder seeming to be the point of contention.

Reviewing Anand's piece, I find this:

Note that we verified that ATI's image quality was the same whether we used NVIDIA's PureVideo DVD decoder or Intervideo's DVD decoder. We chose to benchmark with NVIDIA's PureVideo decoder in order to minimize the number of variables between cards.

I'd like to hear more about the reasoning behind that decision, as it doesn't make sense to me in the same way that, say, insisting on using the same cpu/mobo/memory/OS for both would make sense to me.

Cheng seems to be pointing at the Cat 5.12's as the point when they expect to catch up on tweaking their new toy on the software side, so hopefully someone will revisit Avivo IQ around then.

 

[imported_ST]

Originally posted by: Rollo
I was actually one of the people that was sent that software, I could run the benches and see if I replicate Anands results if anyone cares.

I didn't because I don't have a projector, so I'm not big on watching movies on the smallest screen in the house.

Shoot send it to me and I'll run it and assess it at HDTV (1920x1080p) resolution on my 37" LCD!

 

[rbV5]

It would be nice if Cheng wouldn't throw out statements like when using standard DVD decoders decoders and applications, that AVIVO comes out on top with nothing to back him up.

He even points to the firingsquad review which seems to have results similar to Anand's and they did use Cyberlink decoders which he touts.

For my money, I'm tired of these feature set claims with promises of the future and lackluster support out of the box.

Stop touting features that don't yet exist, and stop making claims that you can't back up with something other than heresay.

I don't disagree with Anand's selection of NV DVD deocders and WMP10 myself. WMP10 is the video engine behind MCE 2005 (Cheng touts MCE and Vista as the "future" as well) and NV DVD decoders are certified by Microsoft as WMP10 and MCE compatable. I use them myself with my ATI cards and they perform well, and I expect AVIVO to work with them as any other decent DVD decoder. I've not heard of any enhanced AVIVO feature support for Intervideo, Cyberlink or any other decoders, so until "something" arrives, you play the cards you got, not the ones you wish you had.

What better place to show your superiority than the competitor's platform against the strength of their solution?

 

[xtknight]

The attitude you get out of this interview is that they obviously don't care about it enough to get it to market. Now we're waiting on MS to implement this interface which they should have done when H.264 was introduced, and ATI should have pushed them to that...stupid. I knew from the day this was announced it would just be another paper launch. What gets me even more is he can't fess up the ATI deinterlacer isn't working right, now. Maximum PC, Anandtech, HQV people, others all agree on it.

BTW, VFW is deader than dead?..get with the new century.

And so is your decode acceleration claims, sir.

I know that some customers were burned last year when a vendor claimed to have a VPU only not to deliver.

And ATI's on the verge of doing the same this year.

 

[geo1]

The FiringSquad piece is awesome. I totally geeked out on that. And, yes, Cheng does point at it himself. I don't get any sense that he's arguing with Anand's findings re PureVideo for what was tested in Anand's piece. But the FS piece does not test Avivo, so I don't see any relation between admiring the FS piece and Cheng's criticisms of Anand's piece for the Avivo findings there (as opposed to the PureVideo findings/score).

 

[rbV5]

Originally posted by: geo1

The FiringSquad piece is awesome. I totally geeked out on that. And, yes, Cheng does point at it himself. I don't get any sense that he's arguing with Anand's findings re PureVideo for what was tested in Anand's piece. But the FS piece does not test Avivo, so I don't see any relation between admiring the FS piece and Cheng's criticisms of Anand's piece for the Avivo findings there (as opposed to the PureVideo findings/score).

FS doesn't dispute the ATI PT findings, and it looks like they had first hand experience to me with Catalyst 5.8, but not Catalyst 5.9 the way I read it.

The irony is that even though NVIDIA's scores were inflated by Principled Technologies, their "key findings" are ones I still agree with completely. Namely, that NVIDIA PureVideo is superior to ATI's Catalyst 5.8, that PureVideo is superior to basic DVD players, and that PureVideo minimizes some of the common artifacts seen with de-interlacing.

I have not yet had the opportunity to test Catalyst 5.9 myself, so I can't make any comment on how it compares to PureVideo but it warrants a closer look

 

[geo1]

Maybe I'm misunderstanding, but my impression is that unlike Anand's comparo, the FS piece is not referring to X1000-class hardware. Certainly the linked .pdf from Principled Technologies specifically refers to R420-based hardware.

Unless the point is that this is entirely a driver/software related feature and that we can reliably assume that X800 would produce the same output as X1000-class so long as the same driver rev is used?

 

[rbV5]

Certainly the linked .pdf from Principled Technologies specifically refers to R420-based hardware.

OIC, you're correct, I thought I saw that the new cards were tested in the pdf.

Unless the point is that this is entirely a driver/software related feature and that we can reliably assume that X800 would produce the same output as X1000-class so long as the same driver rev is used?

That would be a nice comparison as well, but I would hope that the new cards would bring more to the table for deinterlacing than enhanced software support. I do note that Anand only had X1000 hardware 3 points above PT's score for legacy ATI hardware which seems to suggest they're not much better, or perhaps PT gave them additional credit like they did with Purevideo, I doubt the decoders had much to do with the difference, but...? It certainly would be nice to see some more comparisons with both hardware and software.