Yes once Willie Donald floated the eminent domain idiocy the die was cast however what the Irsays did to that team in the 12 years he owned them before moving had nothing to do with the idiots in Annapolis. I was at the 1977 Christmas Eve playoff game against the Oakland Raiders, it was miserable, 20 degrees and windy yet the stadium stayed full to the very end. The fanbase was there which is why the move left such a bitter taste despite the inevitability once the idiots that govern Maryland forced Irsays hand.
Bob Irsay was a moron and was very much like Dan Snyder -- constantly interfering in the team, making player decisions, etc. If he were still alive today, the Colts would probably still suck.
However, in this case, Irsay was right. He tried to work with them on a stadium deal (don't recall if he wanted a new stadium or major improvements to their existing one, no time to look it up) and they refused. IIRC, he was looking at other options (Phoenix and Indianapolis were the finalists) and probably didn't even want to move the team but needed leverage. Once the legislature intervened, he had no choice and Indianapolis had just completed a brand new domed stadium the year before.
As an aside, I still laugh when former Baltimore Colts fans complain about Indianapolis "stealing" their team and then turn right around and cheer on the Ravens, who came from Cleveland. The hypocrisy is comical and the excuses given ("the Ravens left the Browns history in Cleveland!") are a joke.