What controls Turbo Core in Xeons?

[Cata40]

download here

Thank You very much

 

[custom90gt]

So I don't get throttling with ucode 39 but of course it runs at 2.8GHz with AVX loads. Anyone seen throttling with version 27 like that? It won't go over 800mhz...

Also what are you guys setting your current limit to?

 

[Saren]

uCode 27 (0x1F) is the last-known freely-available uCode with no AVX2 clock reduction...

Also, looks like you might not have removed the microcode as intended. What CPU is this? Stock E5-2696 v3 is 2.8GHz all-core turbo...

I am using a pair of E5-2696 v3. BIOS is modded and microcode stripped. 0x3A microcode exists in Linux OS and I can use other versions easily. I will test again later.

 

[kjboughton]

So I don't get throttling with ucode 39 but of course it runs at 2.8GHz with AVX loads. Anyone seen throttling with version 27 like that? It won't go over 800mhz...

Also what are you guys setting your current limit to?

Yes, see what The Stilt previously provided earlier in this thread:
"The newest microcode version for the Haswell-E/EP/EX/EN production stepping (CPUID 0x306F2) is version 0x39 (10/07/2016). This microcode can be used for this exploit, however it will result in lower yield (clocks) than the earlier ones. This microcode is highly recommended if you are satisfied with a more modest boost, or require maximum reliability (professional use). This microcode also has an additional advantage on systems, which lack both the "Power Limit" or "CPU telemetry feature" (SVID) options in the bios. Version 0x39 microcode is one of the few versions, which doesn't feature the bug I call as the "LFM bug". The best way to describe the "LFM bug" is that when you use this exploit, load a newer microcode in flight and then try adjusting any of the CPU parameters (frequency, voltage, power limits, etc), the CPU will lock to the LFM state (typically 800MHz)."

 

[custom90gt]

Yes, see what The Stilt previously provided earlier in this thread:
"The newest microcode version for the Haswell-E/EP/EX/EN production stepping (CPUID 0x306F2) is version 0x39 (10/07/2016). This microcode can be used for this exploit, however it will result in lower yield (clocks) than the earlier ones. This microcode is highly recommended if you are satisfied with a more modest boost, or require maximum reliability (professional use). This microcode also has an additional advantage on systems, which lack both the "Power Limit" or "CPU telemetry feature" (SVID) options in the bios. Version 0x39 microcode is one of the few versions, which doesn't feature the bug I call as the "LFM bug". The best way to describe the "LFM bug" is that when you use this exploit, load a newer microcode in flight and then try adjusting any of the CPU parameters (frequency, voltage, power limits, etc), the CPU will lock to the LFM state (typically 800MHz)."

Interesting, thanks for the info. I wonder what ways around the LFM bugs there are. Maybe it's throttle stop being active that is messing with it?

 

[kjboughton]

Interesting, thanks for the info. I wonder what ways around the LFM bugs there are. Maybe it's throttle stop being active that is messing with it?

TS is obviously sending some commands via OC Mailbox that the CPU doesn't like (uCode bug).

 

[custom90gt]

TS is obviously sending some commands via OC Mailbox that the CPU doesn't like (uCode bug).

Interesting, I'll try disabling it and restart the server.

 

[kjboughton]

RC8 binaries are now up and available for download:
https://github.com/freecableguy/v3x4/releases/tag/v3x4-0.10b-i306f2-rc8

Major changes:
1) Fixed a bug that was preventing PowerCut from being enabled if set. Remember to set VCCIN to Auto in BIOS (if available) when using drivers with PowerCut enabled. Also, if the driver notifies that a reboot is required then simply do a restart as soon as you get to your OS login screen. PowerCut requires a fixed VCCIN and VCCIN can only be set on reset. Hence, PowerCut will require an immediate reboot after any shutdown (not required between successive warm resets).

2) Added back ability to adjust SA voltages. There may be some power savings here but hard to tell if not included in PCU determination to increase boost (probably not). Those with 2S systems will probably care more as this includes the QPI link power budget of which a large part would be the 4.8GHz link from CPU0<->CPU1.

3) Behavior change: set Uncore to max speed and force max speed. This may be a personal choice for some as this does increase idle power and as well may remove power from the power budget that could otherwise be used for the Cores.

4) OC Lock Bit in MSR 0x194 is no longer set. You should now be free to continuing playing with settings post-boot.

 

[custom90gt]

TS is obviously sending some commands via OC Mailbox that the CPU doesn't like (uCode bug).

Thanks for the info btw. I was able to just not launch throttle stop and get 27 to work. The odd thing is I get 2800 points in cinebench with 39 but only 2500 with 27.

 

[Cata40]

download here

It works perfect.
I would have liked to put a picture, but I'm asking for url and I do not know where to load it
I mean it works perfectly
at x34, stressed with bench cpu, the processor still has a lot of power and the temperature does not exceed 50 degrees

 

[kjboughton]

Thanks for the info btw. I was able to just not launch throttle stop and get 27 to work. The odd thing is I get 2800 points in cinebench with 39 but only 2500 with 27.

You can use TS, just be sure to uncheck the box for "Set Multiplier".
You don't want this feature anyway as the CPU will refuse to set anything above the all-core max sustainable frequency. Mean if you are limited (power, current, etc.) to say 3.4GHz, attempting to set 35x or higher will lock the CPU to 34x and keep it there. This would lock out 35x through 38x (for the 2696v3). I am still playing with this.

 

[custom90gt]

You can use TS, just be sure to uncheck the box for "Set Multiplier".
You don't want this feature anyway as the CPU will refuse to set anything above the all-core max sustainable frequency. Mean if you are limited (power, current, etc.) to say 3.4GHz, attempting to set 35x or higher will lock the CPU to 34x and keep it there. This would lock out 35x through 38x (for the 2696v3). I am still playing with this.

Yeah I know what you mean. It's odd that even with TS turned off but started it throttles the CPU.

I get better cinebench scores if I have TS running and the multiplier set to 38 then I do with TS disabled. My guess is it is quicker at ramping up the cores or something like that (even if I have power saving checked as well to keep core speeds low when load is low).

 

[Dufus]

No, supermicro boards do not have any access to voltage settings, only CPU states at most.

BIOS is generic with modifications, this is why I suggested RU.

Well, all I know for sure is that since this thread started, the 2696 v3s I've sold have risen in price (auction and buy now) by over $300US and are still climbing.

Sellers market. Not so good for anyone else.

IMO 2696 no longer has value if you are not going to use it in dual socket systems.

I would agree with that.

So I don't get throttling with ucode 39 but of course it runs at 2.8GHz with AVX loads. Anyone seen throttling with version 27 like that? It won't go over 800mhz...

Also what are you guys setting your current limit to?

0x27 can go into power limiting overdrive and not only throttle speeds to 8x multi but also throw in some duty cycling as well so you may be seeing the equivalent of ~400MHz.

I use 1023 for current limit. Can post my settings if it helps at all but would be somewhat pertinent to my own setup and CPU.

I get better cinebench scores if I have TS running and the multiplier set to 38 then I do with TS disabled.

Have you tried disabling speedstep in the BIOS to check if that's the case?

 

[kjboughton]

I use 1023 for current limit. Can post my settings if it helps at all but would be somewhat pertinent to my own setup and CPU.

Please.

 

[MOF]

Looks like my fun times is over at this point.

My cpu with -50mV,-60mV and -70mV is reaching a 3300mhz,
-80mV an -90mV is reaching 3400mhz but 3400@-90mV not stable in long term.
Up to -90mV my cpu can go 3500mhz but 0.9V too low for 3500mhz. Few second later goes crash.

Looks like 3400mhz@-80mV (with powercut drivers) sweetspot for me at this point.

 

[Dufus]

E5-2683 v3 BIOS settings

OC Tweaker

CPU Config

'Max Core Ratio' needs removing as it is irrelevant for this CPU. Long duration still reverts to 120W but use false reading to bypass this otherwise limited to a RAPL of 40 seconds. Use to run BCLK at 105.1MHZ but saw I was getting a lot of corrected WHEA errors on the PCIe bus with NVMe at of 103MHz and GTX1080 at ~102MHz. Can reduce with lower link speed but just doesn't seem worth it for a few percent.


FIVR Config

Can hit max turbo -1 with AVX2 and -20mV offsets but seems to ramp up over a few seconds from max turbo -2 to -1. -30mV gives much faster response. And yes, VSA is positive.

Voltage Config

Most likely could lower DRAM voltage but haven't bothered. Timings reduced.


V3 Config

OC lock is being used for power reading and not OC lock. Need to change if I get some enthusiasm back. Tjmax doesn't do anything here and some other settings such as "Perf_Plimit_Differential" which if I guess correctly is actual for dual socket and P-State differential between sockets. Maybe I'm guessing wrong.


Advanced CPU

Could add package C2 here but PC2 doesn't seem to make a lot of difference.

Anyway that's what I'm running and works for me but might not work so well for someone else. Needs a fair bit more work yet, maybe some time later on.

Edit: Thanks Cata40, hope imgur works better than Photobucket did

 

[Cata40]

E5-2683 v3 BIOS settings

OC Tweaker

CPU Config

'Max Core Ratio' needs removing as it is irrelevant for this CPU. Long duration still reverts to 120W but use false reading to bypass this otherwise limited to a RAPL of 40 seconds. Use to run BCLK at 105.1MHZ but saw I was getting a lot of corrected WHEA errors on the PCIe bus with NVMe at of 103MHz and GTX1080 at ~102MHz. Can reduce with lower link speed but just doesn't seem worth it for a few percent.


FIVR Config

Can hit max turbo -1 with AVX2 and -20mV offsets but seems to ramp up over a few seconds from max turbo -2 to -1. -30mV gives much faster response. And yes, VSA is positive.

Voltage Config

Most likely could lower DRAM voltage but haven't bothered. Timings reduced.


V3 Config

OC lock is being used for power reading and not OC lock. Need to change if I get some enthusiasm back. Tjmax doesn't do anything here and some other settings such as "Perf_Plimit_Differential" which if I guess correctly is actual for dual socket and P-State differential between sockets. Maybe I'm guessing wrong.


Advanced CPU

Could add package C2 here but PC2 doesn't seem to make a lot of difference.

Anyway that's what I'm running and works for me but might not work so well for someone else. Needs a fair bit more work yet, maybe some time later on.

nothing can be seen in pictures, it's a problem with them

 

[wiseguyuk]

Looks like my fun times is over at this point.

My cpu with -50mV,-60mV and -70mV is reaching a 3300mhz,
-80mV an -90mV is reaching 3400mhz but 3400@-90mV not stable in long term.
Up to -90mV my cpu can go 3500mhz but 0.9V too low for 3500mhz. Few second later goes crash.

Looks like 3400mhz@-80mV (with powercut drivers) sweetspot for me at this point.

Is it -80mV & -50mV ?

Did you compile it yourself ?
Can you provide a link please as I would like to try it.

 

[MOF]

Is it -80mV & -50mV ?

Did you compile it yourself ?
Can you provide a link please as I would like to try it.

kjboughton's latest rc8 drivers OC Lock Bit is open. I put two different efi drives in order

like this;
bcfg driver add 0 fs1:/ v3x4-0.10b-i306f2-rc8_50x1_pc1.9v.efi "first"
bcfg driver add 1 fs1:/ v3_payne_80_50.efi "secondary"



https://peine-braun.net/public_files/XEON_V3_BIOS_MODS/EFI-Drivers/single CPU/

 

[Cata40]

kjboughton's latest rc8 drivers OC Lock Bit is open. I put two different efi drives in order

like this;
bcfg driver add 0 fs1:/ v3x4-0.10b-i306f2-rc8_50x1_pc1.9v.efi "first"
bcfg driver add 1 fs1:/ v3_payne_80_50.efi "secondary"



https://peine-braun.net/public_files/XEON_V3_BIOS_MODS/EFI-Drivers/single CPU/

What it is the result??? Works?

 

[MOF]

Yes, its work but dont do this. Kjboughton will compile native 80-50-pc version for us.

@kjboughton

 

[Cata40]

i have from him -50-65. work great

 

[kjboughton]

Yes, its work but dont do this. Kjboughton will compile native 80-50-pc version for us.

@kjboughton

-80 Core, -50 Uncore, PowerCut (VCCIN = 1.900V) [be sure to set your BIOS to Auto]
download here

 

[Cata40]

-80 Core, -50 Uncore, PowerCut (VCCIN = 1.900V) [be sure to set your BIOS to Auto]
download here

Work great your efi, with -65-50 with no power cut
but at system agent voltage offset, i want -71 not +100
can you do another efi with this value?

 

[kjboughton]

Work great your efi, with -65-50 with no power cut
but at system agent voltage offset, i want -71 not +100
can you do another efi with this value?

-65 mV Core, -50 mv Uncore, -71 mV SA
No PowerCut
download here