I'm basing my opinion that AMD has more moles than we do and probably knew at least the price of the expected GTX 1080 Ti which leads into my next response.
It isn't the product, it was the price. I'm sure AMD already knew what a card like the GTX 1080 Ti could do and probably, like everyone on these forums, expected it to hit between the GTX 1080 and Titan-Pascal in regards to price.
I don't think anyone honestly expected Green for Greed Nvidia would drop the price of the GTX 1080 and slot the 1080 Ti in it's place. I'm sure even AMD didn't expected that. Why I said they probably had intentions of putting Big Vega in the $600 price range, I'd have guess $650, it beats GTX 1080 and wins over enthusiast (because AMD doesn't have that market brand) or causes NV to at least shift prices, but they get to launch at a high price point. With the GTX 1080 Ti @ $700 big Vega has to be a lot closer to it than some what faster than GTX 1080.
With AMD's brand power, they'd be lucky to sell a faster GTX 1080 for $550, but knowing the trends, AMD would be forced to sell a faster card for less, thus putting big Vega in the <$500 price point. And little Vega even lower.
But this is my speculation. I personally hope I'm wrong because AMD can't afford to sell such a die in Vega for less than GP104. It would be suicide.