Flaws on Oxide

[Erenhardt]

Update did nothing to my custom run under D3D. Still 8 fps when they start the shooting. Mantle on the other hand went from 22 to 28 fps in that sequence. CPU utilization is now spread evenly across the cores - around 60% in DX and 50% mantle. That is on Extreme preset.

On low its 35 fps vs 85 fps. CPU utilization 70% vs 99%

I can hear my CPU cooler spinning up to max speed. It never happened outside of proper CPU stress testing.

 

[Gloomy]

http://oxidegames.com/2014/02/04/new-star-swarm-build-better-mantle-performance/

keep the sour grapes flowing

 

[ShintaiDK]

There is a new update again for swarm. The DX performance is like the first update still decreased compared to original.

 

[MutantGith]

http://oxidegames.com/2014/02/04/new-star-swarm-build-better-mantle-performance/

keep the sour grapes flowing

Which sour grapes would those be again?

Now that release implies that it was multi-threaded optimizations that weren't functioning appropriately because of some alterations to the code? Does it seem likely that the CPU usage patterns observable with DX are also impacted, or was it only the Mantle based code path that was adversely effected?

Last time I ran it, it was still using about a half a core on my quad system. But then again, it was also using 20% CPU sitting in the launcher, after they said they had patched that issue, and after I confirmed Steam had downloaded a patch.

I'd be curious to hear what the CPU utilization is with the fix from someone running 14.1

 

[sontin]

With the latest nVidia driver and Deferred Context i get 100% utilization on all four cores of my 2500K. Huge difference to the former driver. On the other hand Deferred Context isn't able to provide the same jump because without it it's has a much better utilization.

 

[Gloomy]

With the latest nVidia driver and Deferred Context i get 100% utilization on all four cores of my 2500K. Huge difference to the former driver. On the other hand Deferred Context isn't able to provide the same jump because without it it's has a much better utilization.

Can you run RTS mode for me and post the results?

 

[sontin]

2500k@4500MHz, Win8.1, 337.50, GTX780TI

===========================================================
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\aheyn_000\Documents\Star Swarm\Output_14_04_08_2226.txt
Version 1.10
04/08/2014 22:26
===========================================================

== Hardware Configuration =================================
GPU:        NVIDIA GeForce GTX 780 Ti 
CPU:        GenuineIntel
               Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Physical Cores:            4
Logical Cores:            4
Physical Memory:         8555474944
Allocatable Memory:        140737488224256
===========================================================


== Configuration ==========================================
API:                DirectX
Scenario:            ScenarioRTS.csv
User Input:            Disabled
Resolution:            1920x1080
Fullscreen:            False
GameCore Update:        16.6 ms
Bloom Quality:            High
PointLight Quality:        High
ToneCurve Quality:        High
Glare Overdraw:            16
Shading Samples:         64
Shade Quality:            Mid
Deferred Contexts:        Enabled
Temporal AA Duration:        16
Temporal AA Time Slice:        2
Detailed Frame Info:        Off
===========================================================


== Results ================================================
Test Duration:            360 Seconds
Total Frames:            9384

Average FPS:            26.07
Average Unit Count:        4170
Maximum Unit Count:        5520
Average Batches/MS:        2355.70
Maximum Batches/MS:        4077.53
Average Batch Count:        87786
Maximum Batch Count:        166410
===========================================================

 

[Paul98]

2500k@4500MHz, Win8.1, 337.50, GTX780TI

===========================================================
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\aheyn_000\Documents\Star Swarm\Output_14_04_08_2226.txt
Version 1.10
04/08/2014 22:26
===========================================================

== Hardware Configuration =================================
GPU:        NVIDIA GeForce GTX 780 Ti 
CPU:        GenuineIntel
               Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Physical Cores:            4
Logical Cores:            4
Physical Memory:         8555474944
Allocatable Memory:        140737488224256
===========================================================


== Configuration ==========================================
API:                DirectX
Scenario:            ScenarioRTS.csv
User Input:            Disabled
Resolution:            1920x1080
Fullscreen:            False
GameCore Update:        16.6 ms
Bloom Quality:            High
PointLight Quality:        High
ToneCurve Quality:        High
Glare Overdraw:            16
Shading Samples:         64
Shade Quality:            Mid
Deferred Contexts:        Enabled
Temporal AA Duration:        16
Temporal AA Time Slice:        2
Detailed Frame Info:        Off
===========================================================


== Results ================================================
Test Duration:            360 Seconds
Total Frames:            9384

Average FPS:            26.07
Average Unit Count:        4170
Maximum Unit Count:        5520
Average Batches/MS:        2355.70
Maximum Batches/MS:        4077.53
Average Batch Count:        87786
Maximum Batch Count:        166410
===========================================================

Your CPU was doing around 100% during that run? Maybe I will try under mantle with my 3570k@ 4ghz, 7850, and windows 7.

 

[Gloomy]

2500k@4500MHz, Win8.1, 337.50, GTX780TI

===========================================================
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\aheyn_000\Documents\Star Swarm\Output_14_04_08_2226.txt
Version 1.10
04/08/2014 22:26
===========================================================

== Hardware Configuration =================================
GPU:        NVIDIA GeForce GTX 780 Ti 
CPU:        GenuineIntel
               Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Physical Cores:            4
Logical Cores:            4
Physical Memory:         8555474944
Allocatable Memory:        140737488224256
===========================================================


== Configuration ==========================================
API:                DirectX
Scenario:            ScenarioRTS.csv
User Input:            Disabled
Resolution:            1920x1080
Fullscreen:            False
GameCore Update:        16.6 ms
Bloom Quality:            High
PointLight Quality:        High
ToneCurve Quality:        High
Glare Overdraw:            16
Shading Samples:         64
Shade Quality:            Mid
Deferred Contexts:        Enabled
Temporal AA Duration:        16
Temporal AA Time Slice:        2
Detailed Frame Info:        Off
===========================================================


== Results ================================================
Test Duration:            360 Seconds
Total Frames:            9384

Average FPS:            26.07
Average Unit Count:        4170
Maximum Unit Count:        5520
Average Batches/MS:        2355.70
Maximum Batches/MS:        4077.53
Average Batch Count:        87786
Maximum Batch Count:        166410
===========================================================

That is extremely good and better than I expected. Not as good as Mantle, but much closer than it was on this test before.

 

[Paul98]

I got 27 fps avg on my run under mantle with around 30% cpu utilization. using 3570k@ 4ghz, 7850, and windows 7.

 

[Gloomy]

Here's what I get:

===========================================================
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\Shoes\Documents\Star Swarm\Output_14_04_07_1211.txt
Version 1.10
04/07/2014 12:11
===========================================================

== Hardware Configuration =================================
GPU:        AMD Radeon R9 200 Series
CPU:        GenuineIntel
        Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
Physical Cores:            4
Logical Cores:            8
Physical Memory:         8455278592
Allocatable Memory:        140737488224256
===========================================================


== Configuration ==========================================
API:                Mantle
Scenario:            ScenarioRTS.csv
User Input:            Disabled
Resolution:            1920x1080
Fullscreen:            True
GameCore Update:        16.6 ms
Bloom Quality:            High
PointLight Quality:        High
ToneCurve Quality:        High
Glare Overdraw:            16
Shading Samples:         64
Shade Quality:            Mid
Deferred Contexts:        Disabled
Temporal AA Duration:        16
Temporal AA Time Slice:        2
Detailed Frame Info:        C:\Users\Shoes\Documents\Star Swarm\FrameDump_14_04_07_1211.csv
===========================================================


== Results ================================================
Test Duration:            360 Seconds
Total Frames:            12326

Average FPS:            34.24
Average Unit Count:        4349
Maximum Unit Count:        5546
Average Batches/MS:        2903.06
Maximum Batches/MS:        4091.06
Average Batch Count:        85811
Maximum Batch Count:        164516
===========================================================

I'll monitor utilization later.

 

[Carfax83]

PCPer did a performance comparison between Mantle and the 337.50 drivers in Star Swarm..

Mantle has superior scaling as you would think, but it's still amazing what NVidia has been able to accomplish with their new driver with regular old DX11 in a benchmark purposely designed to make Mantle look good..

 

[3DVagabond]

PCPer did a performance comparison between Mantle and the 337.50 drivers in Star Swarm..

Mantle has superior scaling as you would think, but it's still amazing what NVidia has been able to accomplish with their new driver with regular old DX11 in a benchmark purposely designed to make Mantle look good..

It's obvious that they've done the work to make the DX path "look good" as well. If they didn't then nothing nVidia or Microsoft or anyone else did would matter. Just shows that all the fear mongering about fragmenting the market and the effort put into Mantle would detract from the optimizations for DX was nothing but hype and spin. Some companies don't subscribe to anti competitive business practices like that.

 

[Sequences]

If you look at the raw data, D3D is much more optimized than Mantle.

 

[Enigmoid]

If you look at the raw data, D3D is much more optimized than Mantle.

Where is this?

 

[Sequences]

Where is this?

I'm traveling right now, so I don't have the raw data with me (I ran the experiments myself). However, I can tell you how I arrived at the conclusion, and you can verify if you wish.

My configuration:
AMD PhenomII X6 1090T
8GB RAM
HD7850 2GB, stock setting of 860/1200 I believe.
Windows 7 SP1
Latest AMD drivers, 14.4 (the one that was just release, not beta)
3600x1920 eyefinity

I ran through Star Swarm with max detail at my resolution with raw data output enabled. After the test, I plotted batch count vs millisecond time to do the batch. Compare Mantle chart against D3D chart.

I found that Mantle scales very well with high batch counts while D3D does not. However, the behavior of each of these APIs reveals their level of optimization. Mantle response timing scales very well when it comes to increased batch counts, being significantly faster than D3D towards the higher batch counts. Towards lower batches, however, their response times are similar. What's interesting is how tightly banded the D3D timings are with respect to batches. Mantle response timings are generally uniformly distributed along the same range with respect to batch counts, making it seem like it is independent of batch counts.

I'm not sure if you have done deep dive on application performance tunings before, but tight bands indicates optimization. It is very rare for something untuned and unoptimized to have such tight banding out of the box.

I plan on tuning Mantle runs a bit more, if possible. I did notice that Mantle spread the load evenly between my 6 cores during the run while D3D only pushed 2 cores. While it is unlikely for context switching to cause jitter at the millisecond level, this is something that needs to be verified.

 

[Enigmoid]

I'm traveling right now, so I don't have the raw data with me (I ran the experiments myself). However, I can tell you how I arrived at the conclusion, and you can verify if you wish.

My configuration:
AMD PhenomII X6 1090T
8GB RAM
HD7850 2GB, stock setting of 860/1200 I believe.
Windows 7 SP1
Latest AMD drivers, 14.4 (the one that was just release, not beta)
3600x1920 eyefinity

I ran through Star Swarm with max detail at my resolution with raw data output enabled. After the test, I plotted batch count vs millisecond time to do the batch. Compare Mantle chart against D3D chart.

I found that Mantle scales very well with high batch counts while D3D does not. However, the behavior of each of these APIs reveals their level of optimization. Mantle response timing scales very well when it comes to increased batch counts, being significantly faster than D3D towards the higher batch counts. Towards lower batches, however, their response times are similar. What's interesting is how tightly banded the D3D timings are with respect to batches. Mantle response timings are generally uniformly distributed along the same range with respect to batch counts, making it seem like it is independent of batch counts.

I'm not sure if you have done deep dive on application performance tunings before, but tight bands indicates optimization. It is very rare for something untuned and unoptimized to have such tight banding out of the box.

I plan on tuning Mantle runs a bit more, if possible. I did notice that Mantle spread the load evenly between my 6 cores during the run while D3D only pushed 2 cores. While it is unlikely for context switching to cause jitter at the millisecond level, this is something that needs to be verified.

You haven't tried anything with a Nvidia card?

Personally, I don't see any proof for the claim that D3D is better optimized. Mantle is specifically designed for GCN; that itself is a huge optimization.

 

[Alatar]

It's obvious that they've done the work to make the DX path "look good" as well. If they didn't then nothing nVidia or Microsoft or anyone else did would matter. Just shows that all the fear mongering about fragmenting the market and the effort put into Mantle would detract from the optimizations for DX was nothing but hype and spin. Some companies don't subscribe to anti competitive business practices like that.

What?

When the only thing you change is Nvidia's drivers (from 335 to 337) you see huge gains in the same version of Star Swarm.

The star swarm improvements on NV hardware have absolutely nothing to do with the Oxide engine and its updates.

Back when the 337.50s launched I compared 3 driver sets in SS custom scenario (same camera angle all the time, same ship spawns) and the difference was night and day:

Star Swarm still runs absolutely terrible on older NV drivers and current AMD drivers when using DX11.

link to detailed data

They're DX11 wide improvements. You see gains pretty much every time you're CPU limited. BF4 multi for example shows nice gains when you start looking into these drivers more than the "single player 1440p testing on stock GPUs" done by review sites.

 

[Sequences]

You haven't tried anything with a Nvidia card?

I don't have a card from the green team. I make do with what I have.

Personally, I don't see any proof for the claim that D3D is better optimized. Mantle is specifically designed for GCN; that itself is a huge optimization.

When I talk about optimization of D3D and Mantle, I'm referring to how well the application uses the API to do what it needs to do. When looking at the tight bands showing response time of D3D with respect to batch jobs, you see how predictable it is. A well tuned library will give you predictable results. Mantle, on the other hand, is much more scattered in comparison, which is evidence to me that Star Swarm does not use it as well as it uses D3D. Since the test allows us to run similar tests between both D3D and Mantle and I see tight bands for D3D, I conclude that the scattering is due to the API itself, as opposed to the Oxide Engine. This is enough evidence for me to conclude that D3D is more optimized internally than Mantle is internally. It should make sense, as D3D has been released much earlier and has been through much more than Mantle.

On Star Swarm's release, Oxide had said that they spent more time optimizing for D3D than Mantle (pardon my paraphrase). I am going to assume that this is what they meant.

 

[Paul98]

It was built from the ground up for DX11 then ported to mantle.

Any word on that CPU utilization?

 

[VulgarDisplay]

I don't have a card from the green team. I make do with what I have.



When I talk about optimization of D3D and Mantle, I'm referring to how well the application uses the API to do what it needs to do. When looking at the tight bands showing response time of D3D with respect to batch jobs, you see how predictable it is. A well tuned library will give you predictable results. Mantle, on the other hand, is much more scattered in comparison, which is evidence to me that Star Swarm does not use it as well as it uses D3D. Since the test allows us to run similar tests between both D3D and Mantle and I see tight bands for D3D, I conclude that the scattering is due to the API itself, as opposed to the Oxide Engine. This is enough evidence for me to conclude that D3D is more optimized internally than Mantle is internally. It should make sense, as D3D has been released much earlier and has been through much more than Mantle.

On Star Swarm's release, Oxide had said that they spent more time optimizing for D3D than Mantle (pardon my paraphrase). I am going to assume that this is what they meant.

It sounds to me more like mantle is doing the work immediately when it is requested and dx11 is barfing up chunks of what it can as the CPU finishes.

 

[3DVagabond]

What?

When the only thing you change is Nvidia's drivers (from 335 to 337) you see huge gains in the same version of Star Swarm.

The star swarm improvements on NV hardware have absolutely nothing to do with the Oxide engine and its updates.

Back when the 337.50s launched I compared 3 driver sets in SS custom scenario (same camera angle all the time, same ship spawns) and the difference was night and day:

Star Swarm still runs absolutely terrible on older NV drivers and current AMD drivers when using DX11.

link to detailed data

They're DX11 wide improvements. You see gains pretty much every time you're CPU limited. BF4 multi for example shows nice gains when you start looking into these drivers more than the "single player 1440p testing on stock GPUs" done by review sites.

Reread what I said.

It's obvious that they've done the work to make the DX path "look good" as well. If they didn't then nothing nVidia or Microsoft or anyone else did would matter. Just shows that all the fear mongering about fragmenting the market and the effort put into Mantle would detract from the optimizations for DX was nothing but hype and spin. Some companies don't subscribe to anti competitive business practices like that.

Your response has nothing to do with my statement.

If the game engine was not well optimized for DX then it wouldn't matter what nVidia did the DX performance would suck. Drivers can't fix a broken engine/game. I never said nVidia hasn't optimized their drivers. I wasn't talking about nVidia at all. So, please reread what I said, because you completely missed the point. I'll highlight the important part.

On whet you said though, It shows me how important it is for every single game to be driver optimized to extract the best performance out of it. Not so with Mantle. It's the game that's optimized to the API rather than the drivers having to be redone for every game.

 

[Shivansps]

These are my results with a GTX750 TI and 337.50.

===========================================================
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\Shivan\Documents\Star Swarm\Output_14_05_25_2221.txt
Version 1.10
05/25/2014 22:21
===========================================================

== Hardware Configuration =================================
GPU: NVIDIA GeForce GTX 750 Ti
CPU: GenuineIntel
Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Physical Cores: 4
Logical Cores: 4
Physical Memory: 10716540928
Allocatable Memory: 8796092891136
===========================================================


== Configuration ==========================================
API: DirectX
Scenario: ScenarioFollow.csv
User Input: Disabled
Resolution: 1920x1080
Fullscreen: True
GameCore Update: 16.6 ms
Bloom Quality: High
PointLight Quality: High
ToneCurve Quality: High
Glare Overdraw: 16
Shading Samples: 64
Shade Quality: Mid
Deferred Contexts: Enabled
Temporal AA Duration: 16
Temporal AA Time Slice: 2
Detailed Frame Info: Off
===========================================================


== Results ================================================
Test Duration: 360 Seconds
Total Frames: 14918

Average FPS: 41.44
Average Unit Count: 4320
Maximum Unit Count: 5316
Average Batches/MS: 877.84
Maximum Batches/MS: 3090.41
Average Batch Count: 23581
Maximum Batch Count: 103044
===========================================================

Motion blur off

===========================================================
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\Shivan\Documents\Star Swarm\Output_14_05_25_2227.txt
Version 1.10
05/25/2014 22:27
===========================================================

== Hardware Configuration =================================
GPU: NVIDIA GeForce GTX 750 Ti
CPU: GenuineIntel
Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
Physical Cores: 4
Logical Cores: 4
Physical Memory: 10716540928
Allocatable Memory: 8796092891136
===========================================================


== Configuration ==========================================
API: DirectX
Scenario: ScenarioFollow.csv
User Input: Disabled
Resolution: 1920x1080
Fullscreen: True
GameCore Update: 16.6 ms
Bloom Quality: High
PointLight Quality: High
ToneCurve Quality: High
Glare Overdraw: 16
Shading Samples: 64
Shade Quality: Mid
Deferred Contexts: Enabled
Temporal AA Duration: 16
Temporal AA Time Slice: 2
Detailed Frame Info: Off
===========================================================


== Results ================================================
Test Duration: 360 Seconds
Total Frames: 25154

Average FPS: 69.87
Average Unit Count: 4539
Maximum Unit Count: 5708
Average Batches/MS: 276.69
Maximum Batches/MS: 1479.56
Average Batch Count: 4133
Maximum Batch Count: 27322
===========================================================

 

[desprado]

and yet you keep ignoring things like this:


http://www.slideshare.net/DICEStudio/directx-11-rendering-in-battlefield-3
https://twitter.com/repi


but you know honest representation of the situation isn't something your actually after right .


who cares if AMD DX11 isn't multi threaded when it doesn't actually buy you anything........

so NV supports DCL, so does frostbite and yet mantel still shows massive performance improvements above and beyond it....

Which Massive improvements u are talking? I have R9 290 CF and Mantle has made BF4 worst and instability.

 

[2is]

You might want to read up and or watch the interviews with the Oxide devs on what their goals were with their engine, how its engineered, and what is actually going on that you are looking at.

The short is that they were going for a render engine that is exactly like what Hollywood uses in movies only it's real time. Everything on the screen is an independent entity with full physics and AI were applicable.

http://forums.anandtech.com/showthread.php?t=2361756

https://www.youtube.com/watch?v=VC8RWntPRvI

Someone's drinking the Kool-Aid...