How big is the processor's execution window for code optimization?

[chrstrbrts]

Hello,

OK, here's what I mean.

Modern processors have a multitude of expedition and optimization mechanisms.

Some examples are: branch prediction, out-of-order instruction execution, pre-fetching instructions.

But how broad is the processor's scope when it performs these optimizations?

Let's say a big process is loaded into memory and an entire 2 MB page is dedicated to its code.

Does the processor look at the entire page and optimize the entire page before beginning execution?

Or does it just start at the code entry point and only look at the next 10 or 15 instructions and optimize within that small window?

 

[Cogman]

https://www.microway.com/knowledge-...intel-xeon-e5-2600v4-broadwell-ep-processors/

So no it isn't doing the whole program.

How much is done depends entirely on both the code being executed and the processor it is executing on.

For intel CPUs, they have a scheduler of micro-opts. They build out the dependency tree and then execute things as blocks (waiting on IO) are removed. According to the above article, their modern CPUs can handle 64 entries.

I imagine that now because we have stopped getting the free performance (in the form of larger caches primarily) and power decreases afforded by node shrinks, that what the CPU will optimize will become ever more impressive.

 

[chrstrbrts]

So no it isn't doing the whole program. According to the above article, their modern CPUs can handle 64 entries.

OK, that makes sense.

Trying to optimize possibly millions of instructions in one shot seemed woefully impractical to me; I suspected that the scope was much smaller.

Thank you, sir.

 

[Greyguy1948]

https://www.microway.com/knowledge-...intel-xeon-e5-2600v4-broadwell-ep-processors/

So no it isn't doing the whole program.

How much is done depends entirely on both the code being executed and the processor it is executing on.

For intel CPUs, they have a scheduler of micro-opts. They build out the dependency tree and then execute things as blocks (waiting on IO) are removed. According to the above article, their modern CPUs can handle 64 entries.

I imagine that now because we have stopped getting the free performance (in the form of larger caches primarily) and power decreases afforded by node shrinks, that what the CPU will optimize will become ever more impressive.

Interesting subject!
I can see that in SPEC the step from base to peak is very small for Intel compiler compared to other compilers. Is this a sign of better prediction?

 

[Cogman]

Hard to say. Unfortunately (fortunately?) as more and more optimizations enter both the hardware and software level, it becomes harder and harder to tell which optimizations are due to better compilers, which are due to better processors, and what exactly is triggering the fast path.

intel certainly hasn't been shy of using their compiler to aggressively optimize for their CPUs. Their compiler's have also traditionally done a better job of compiling code than pretty much everyone else out there. In particular, they have led the pack when it comes to things like autovectorization (the automatic use of instructions like SSE and AVX).

 

[exdeath]

The scope only goes so far as there are available execution units, rename registers, reservation stations, and pipeline stages. The entire point is simply to keep all resources doing something every clock.

It's not about optimizing code, it's about the decode and schedule unit avoiding stalls and keeping all resources actively doing something by keeping all pipeline stages filled and moving.

See Tomasulo’s algorithm, register renaming, and reservation stations, these are the heart of every modern microprocessor.

Branch prediction is semi-unrelated, it's more a MMU task necessary because of the huge gap between DRAM and CPU speed and deep pipelines. Obviously there is communication and cooperation with the control logic as conditional branches hit the execution stage but branch hits and misses can be thought of as being forwarded to the MMU / prefetcher more than anything.