The history of it as I remember was HyperThreading was added as a fix to a issue of excessivly long pipelines present in the P4's. The performance the cpu suffered with a branch miss prediction was quite massive, enough to make a P4 less useful than previous cpus like the P3.
It was not included in the original Core processors as intel went back to the drawing board and left the development of the P4 development line for dead (it was intended to push the Ghz limit and did not care about anything else).
What intel did was went back and started developed again from the Pentium M(designed for laptops) range of processors. Side note, but one of the major advantages of the Pentium M over ealier Pentiums was the ability to power gate unused cache, leading to massive power savings).
With the shorter pipe line of the cores, hyperthreading was point less (and with short development time) had even less reason to use it in the inital core designs.
It was latter brought back as it did have some use, but performance from it was really hit or miss with some bentchmarks showing a performance loss with it enabled and others showing such a small increase as to be more of a "what the" moment.
The current implementation of Hyperthreading is better, but while it is not as bad as it was, still can show the odd bentchmark performing better without it.
Currently it is used for marketing segmentation (ie: i3's have it, i5's don't, i7's do). But then it is used by marketing to "upsell" a processor to the masses without the cost of actually having a proper cpu behind what windows reports as the number of processors present.
Personally, the rule of thumb when it comes to "do I need hyperthreading" as I see it is still no.
On the topic of clock multiplier locking, Intel started doing it as a rather large underground market developed about the 1Ghz era, back when intel's quality was excellent for overclockers. Back when buying a 300Mhz part and getting something like 99% of all cpus overclocking to 450Mhz without changing voltages. Due to the better silicon, businesses existed to re-badge the intel cpus (buy the lowest/cheapest, cut back the top of the cpu and re-laser etch the details for a faster CPU back on). Only probably $200-300 increase in wholesale prices (between the cheap and the one they re-badged to), but the numbers involved were massive. The fix from intel to kill this problem off was to burn / lock the maximum multiplier a given chip was allowed to use. So leaving the only way to overclock to adjust the FSB (intel at least left the lower multipliers unlocked to allow some flexability for the overclocking market).
AMD did not have this problem (so left there multipliers unlocked as a selling feature) as their name did not attract the same issues from rebadges, price differences from cheap to high end was far narrower and their silicon, in comparison at the time, was of a far lower quality so failure rates from this practice was far higher.