As a software engineer who writes programs compile time is very important. If I am working, I attempt to fix some error then I wait for a recompile (could be 1 min to 2 hours depending on what I need to recompile, but mostly a few mins) then check the fix, re-compile again, etc. I don't need check emails (do that first thing) or have time to go to a meeting (which I couldn't schedule around my compilation anyway), nor can I really fix something else.
Sure, I also want code to compile ASAP. But 5% faster is not enough to matter. Getting more RAM, an SSD, or more cores resulting in perhaps 50-100% speedup is something else though. Then it gets noticeable.
And everyone has their way-of-working. But most people do not sit idle for 2 hours waiting for a compile to finish. There's always other stuff to do, at least where I've worked. Often you have several ongoing tasks in parallel, you work on multiple branches, you need to investigate some code, send emails, discuss issues with colleagues, need to read up on a subject, etc. But maybe things are different where you work.
My current work machine has 12 slow cpu cores as that's what someone in IT thought was best. They were wrong. A lot of the time I am held up by that slow single threaded speed. Running stuff mostly comes down to single threaded performance, compilation can on occasion use all the cores but then you have to do some huge link which is single threaded again. I would take 6 fast cores over 12 slow ones. What is more key is having a lot of memory as then windows caches everything which saves a lot of disk accesses.
If you're having 12 cores then you might run into I/O as a bottleneck, unless you have plenty of RAM and/or a fast SSD as you said.
But in general building software usually benefits hugely from multiple cores (and RAM & fast I/O). Sure there are some phases like linking that you mentioned that are still single threaded. But if properly designed with libs and reduction of dependencies across modules it should not be a huge part of the total build time. Especially not when doing a complete re-build, then linking should not take most of the time. And complete rebuilds are what takes a long time (sometimes hours). When only recompiling a couple of edited source code files the linkage time will take up a larger percentage of the total build time, but the total time for a build is much less (often seconds or minutes, unless some commonly shared file(s) are edited). So 5% increased link time is really not much IMHO.