Intel has likely had 6 and 8 core designs in the labs for quite a while. The LACK of introducing them had nothing to do with AMD, it all had to do with margins. However, with Ryzen, Intel finally had to up their game a bit. It would take a fool not to see this. There is no way Intel would have released coffee lake as 6 cores had AMD not released Ryzen. When AMD regained the performance crown (ignore overclocking for a minute, that is only a small, but important niche of buyers), Intel had to do something to react. The 1800X basically became one of the fastest chips you could buy and leadership at Intel likely decided to release 6 cores starting with coffee lake to counter that. I would have done the same thing honestly. why double up on transistor count and cut margins in half if you don't have to? You can also bet your bottom dollar that Intel is working on some real IPC improvements behind closed doors along with true 8+ core designs for the future. When AMD releases the 2700X, Intel will again need to respond. Remember, this competition is good for consumers. Also remember that overclockers and tech junkies are a small subset of a much larger market. Intel doesn't want every Dell PC out there to have the AMD Ryzen logo on it, so they now have to make sure they have CPU designs that don't look like bulldozer did back in the day.
Many likely dispute this viewpoint, but I've been in the tech scene a long time. Intel has been doing the same thing since the 386 was released. Incremental improvements each year, sometimes you'll get some new technology when a competitor (Cyrix or AMD back in the day) leapfrogs you, sometimes you would get a new architecture (Pentium, Pentium Pro). Eventually competitors would be unable to keep up and Intel would slow down again. Then someone would step up like AMD did with the K6-2/K6-3 and then Athlon/Athlon 64 (which kicked the living shit out of Intel in the P3/P4 days) and Intel would have to up the game again (The 'core' design was actually based off the old pentium pro design if I recall correctly). It is only a matter of time before AMD releases the Ryzen 3xxx series and Intel will have to have a new architecture or a heavily revamped one to compete. (I like to compare Ryzen with the K6-2 and Ryzen 2xxxx with the K6-3, but with a bit more OOMPH to them, since they compete very well). They are likely reworking a future release and working HARD on 10nm. . They are also working on discrete GPUs to help raise revenue/margins to counter the competition that AMD has brought to the market.
The next few years are likely to bring no shortage of surprises, that is for sure. Once AMD gets going on 7nm you'll see a new flagship GPU, a new flagship CPU, and then you'll see Intel push out a 10nm Flagship CPU and GPU to compete. There are also others with licenses for the x86 tech, and some patents have expired, so we could see other competitors in the future, and then you have ARM, Nvidia, and Windows on ARM. Microsoft Windows and Microsoft Office will be available for ARM CPUs at some point in the future as well (and I mean FULL office), so you'll see some shift in that direction.