(Discussion) Futuremark 3DMark Time Spy Directx 12 Benchmark

[Silverforce11]

The graphics pipeline is serial. Rasterizers are idling because running workload has not or had reached them. You cant access these Rasterizers through a dedicated queue to fill them with "work". You should read a little bit more about this: https://msdn.microsoft.com/en-us/library/windows/desktop/ff476882(v=vs.85).aspx

DMA engines copy information from the host system to the gpu before the GPU starts the workload. It has nothing to do with "asynchronous shaders(compute)".

The graphics pipeline USED to be serial.

id Software run Shadow Maps on the Rasterizer and other tasks on the Shaders at the same time.. you say they can't?!

Shadow map rendering, as an example, is typically bottlenecked by fixed pipeline processing (eg rasterisation) and memory bandwidth rather than raw compute performance. This means that when rendering your shadow maps, if nothing is running in parallel, you're effectively wasting a lot of GPU processing power.

/facepalm

In simple to understand diagrams, watch and learn:

https://youtu.be/H1L4iLIU9xU?t=14m49s

Yes you can, access Rasterizers & DMAs to fill them with "work" while Shaders are running. That's the entire point of the Multi-Engine design of DX12/Vulkan.

 

[Thala]

id Software run Shadow Maps on the Rasterizer and other tasks on the Shaders at the same time.. you say they can't?!

I think was Sontin is trying to say is, that Shadow Maps is a graphics task and you run a compute task in parallel to utilize the shaders.

 

[sontin]

I think was Sontin is trying to say is, that Shadow Maps is a graphics task and you run a compute task in parallel to utilize the shaders.

And id is saying the opposite to Silverforce:

Shadow map rendering, as an example, is typically bottlenecked by fixed pipeline processing (eg rasterisation) and memory bandwidth rather than raw compute performance.

So when the rasterizer stage is limiting the performance they are using Asynchronous Shaders to utilize the compute units. Futuremark has the same approach.

Claiming that Futuremark should use the rasterizer stage along with "shaders" is just a pointless observation.

 

[Elixer]

I genuinely don't understand why people are spending so much time trying to argue so vehemently that "AMD's async is better than NVIDIA's."

All that matters is delivered performance. It doesn't really matter to gamers how an IHV gets there.

That is not what people are (or should) be arguing about.
The issue here is, they (3Dmark) are purposely neutering what a full DX12 / Vulkan engine can actually do, to appease a vendor who uses drivers to get around their lack of hardware functions.
Since they must do it in drivers, that introduces expensive context switching, so, obviously, they (nvidia) don't want to do that, since it shows their hardware severely under performing the competition. (BTW, hardware context switching is faster than software, but you still get a penalty of doing it for "half support".)

As was previously shown in this thread, the Fury card is faster than Nvidia's newest lineup using a fairly well optimized Vulkan game (Doom), and that is a hard pill to swallow for people who just dropped down $500+ for their newest toy.

So, once again, it proves that 3Dmark is basically worthless as a real world test to show what a full implementation of DX12/Vulkan can do.

 

[Red Hawk]

That is not what people are (or should) be arguing about.
The issue here is, they (3Dmark) are purposely neutering what a full DX12 / Vulkan engine can actually do, to appease a vendor who uses drivers to get around their lack of hardware functions.
Since they must do it in drivers, that introduces expensive context switching, so, obviously, they (nvidia) don't want to do that, since it shows their hardware severely under performing the competition. (BTW, hardware context switching is faster than software, but you still get a penalty of doing it for "half support".)

As was previously shown in this thread, the Fury card is faster than Nvidia's newest lineup using a fairly well optimized Vulkan game (Doom), and that is a hard pill to swallow for people who just dropped down $500+ for their newest toy.

So, once again, it proves that 3Dmark is basically worthless as a real world test to show what a full implementation of DX12/Vulkan can do.

Your "issue" is tin foil hat theorizing and speculation. There's no proof that async compute was neutered to appease Nvidia; in fact there's no hard proof that async compute was neutered at all, just speculation. Would you support it if Time Spy was rendering with a full FL12.1 codepath and thus using features only Nvidia cards support even if it meant Nvidia still had the performance advantage? It's alright to talk about this, but it's kind of presumptuous to act like anything's been proven.

 

[FM_Jarnis]

I have a question, does 3DMark Time Spy have a FL 12_0 code path ?

No, it has a single DX12 FL 11_0 code path.

 

[FM_Jarnis]

-License or work with the teams behind the most widespread/popular game engines (Frostbyte & Cryengine for example) to create a capsulized, repeatable benchmark built on engines that people will actually use. You can even launch separate benchmarks based on separate engines.

We've actually discussed about the option of using a third party graphics engine for a 3DMark test - our artists at least would sure love the more mature art pipelines of full game engines - but our benchmark development program members have indicated that it would reduce the usefulness of 3DMark to them. "If you want to benchmark with a game engine, run a game made using that engine."

3DMark Time Spy engine is specifically written to be a neutral, "reference implementation" engine for DX12 FL11_0.

 

[sontin]

FM_Jarnis,
is there a reason why Ansiotropic Filtering is disabled for the benchmark? Seems a little bit odd...

 

[FM_Jarnis]

FM_Jarnis,
is there a reason why Ansiotropic Filtering is disabled for the benchmark? Seems a little bit odd...

I honestly don't know how the default settings relating filtering and AA were selected, but I'm sure the dev team had a reason to choose what they are.

Custom mode allows you to change those.

 

[Hitman928]

Jarnis, I just want to thank you for coming in here to contribute to the thread. I know it can be a little hostile at times, but your input is very welcome by many.

 

[3DVagabond]

Jarnis, I just want to thank you for coming in here to contribute to the thread. I know it can be a little hostile at times, but your input is very welcome by many.

Seconded.

 

[FM_Jarnis]

Jarnis, I just want to thank you for coming in here to contribute to the thread. I know it can be a little hostile at times, but your input is very welcome by many.

I do mostly tech support, so thick skin and flame-retardant pants are part of the job description.

 

[trinibwoy]

The issue here is, they (3Dmark) are purposely neutering what a full DX12 / Vulkan engine can actually do, to appease a vendor who uses drivers to get around their lack of hardware functions.

Is there even a single shred of evidence to support that claim?

I do mostly tech support, so thick skin and flame-retardant pants are part of the job description.

You're a saint to even bother replying to this nonsense. Seriously.

 

[Dygaza]

Using no AF is actually interesting decision. I just ran test with 16x AF, and I lost like 0.5 fps in gametest 2. So it doesn't really have a real inpact on performance either.

 

[Dygaza]

There has been talks about heavy tesselation, especially on graphics test 2. I want to debunk that simply with my own test.

Fury X @ stock

Max tesselation factor / Tesselation factor scale /fps

1(0) 28,31 fps
8(1) 27,89 fps
16(1) 27,65 fps
32(1) 27,54 fps (default)
64(1) 27,15 fps

 

[guskline]

FM_Jarnis: As a fairly long time poster, a big thank you for jumping on this forum and taking the time to explain TimeSpy.

 

[provost]

We've actually discussed about the option of using a third party graphics engine for a 3DMark test - our artists at least would sure love the more mature art pipelines of full game engines - but our benchmark development program members have indicated that it would reduce the usefulness of 3DMark to them. "If you want to benchmark with a game engine, run a game made using that engine."

3DMark Time Spy engine is specifically written to be a neutral, "reference implementation" engine for DX12 FL11_0.

DX12 FL11_0 implementation may be "neutral" in maintaining the status quo...lol Here is to hoping that the game developers will take the path of least resistance with vulkan and/or DX 12 to leverage the full benefits of a low level API to port highly optimized games without the need of much middleware meddling, i.e Doom being a good example of "reference implementation" for a low level api, imho...lol

 

[Arachnotronic]

Jarnis, I just want to thank you for coming in here to contribute to the thread. I know it can be a little hostile at times, but your input is very welcome by many.

Yep, hope he sticks around. :thumbsup:

 

[railven]

Yep, hope he sticks around. :thumbsup:

Welps he pretty much made what I said on page 1 more "official" if you will. Anyone who thinks the majority of game devs aren't going to focus on the lowest common denominator must only play AAA-sponsored games.

Again, some of the games I've been playing don't even have DX11 features. I'm greatly enjoying the influx of once only console exclusives to PC. Xbone/PS4/Wii-U are pretty much only brought out for those rare gems. PC rules the roost in mi casa.

 

[FM_Jarnis]

Using no AF is actually interesting decision. I just ran test with 16x AF, and I lost like 0.5 fps in gametest 2. So it doesn't really have a real inpact on performance either.

Remember; this has to run even on Intel Haswell integrated... Try the same thing there

 

[Dygaza]

Remember; this has to run even on Intel Haswell integrated... Try the same thing there

Fair point. It's sometimes hard to see things outside of your own biased view.

 

[Azix]

Welps he pretty much made what I said on page 1 more "official" if you will. Anyone who thinks the majority of game devs aren't going to focus on the lowest common denominator must only play AAA-sponsored games.

Again, some of the games I've been playing don't even have DX11 features. I'm greatly enjoying the influx of once only console exclusives to PC. Xbone/PS4/Wii-U are pretty much only brought out for those rare gems. PC rules the roost in mi casa.

Those games don't matter. Unless they actually end up that demanding. Its the games that will push your hardware to tears that these discussions are relevant to.

 

[Red Hawk]

There has been talks about heavy tesselation, especially on graphics test 2. I want to debunk that simply with my own test.

Fury X @ stock

Max tesselation factor / Tesselation factor scale /fps

1(0) 28,31 fps
8(1) 27,89 fps
16(1) 27,65 fps
32(1) 27,54 fps (default)
64(1) 27,15 fps

Oh there's no question the benchmark uses a lot of tessellation. 3DMark says so themselves in their technical guide.

Well over twice as many triangles in Time Spy Graphics Test 2 as Fire Strike Graphics Test 2, and almost five times as many tessellation patches between Time Spy Graphics 2 and Fire Strike Graphics Test 1. The reason that performance doesn't tank quite as much as that might suggest is supposedly because 3DMark's DirectX 12 engine is better at managing geometry than its DirectX 11 engine was. Your Fury X not losing any frames at an already relatively low frame rate is just indicative of its tessellation hardware holding up and the bottleneck coming from elsewhere (like those compute shader invocations, good lord).

 

[Dygaza]

My bad for not being clear. I'm not disputing the fact that there is a lot of tesselation. Just the claim that AMD cards are suffering from it. As clearly it's not holding them back, atleast in these framerates yet.

 

[FM_Jarnis]

DX12 can handle a ton more geometry - it is one of the major features of it - so we put a ton of geometry in the scenes. That was one of the reasons for the "Museum" theme - lots of transparent display cases with massive number of unique objects. We also raided our old assets fairly extensively and touched them up for physically based rendering to push up the scene complexity.

At the start of the Demo you can see Fire Strike (okay, a piece of it, with some simplifications, but the geometry of the area that is in the case is the same) and it continues running in the display case all trough the demo, just small part of the whole scene. The demo also has *two* museums and you see the "past" version in the time scope throughout the scene.

AMD hasn't had a problem with large amount of tessellation for several years. You're thinking of very old cards...