A lot of these high interest banks are s--t banks that, if congress weren't in the pockets of the banking lobby, wouldn't qualify for FDIC insurance. If the economy goes into a tailspin, these are the banks that will collapse, taking your tax money with them as the FDIC bails them out.
Why do you care? Well, if you're a small investor with <$100k, you don't--too much. There is the little matter of the lag time between bank failure and FDIC reimbursement. This can be a substantial amount of time, so if you need your money right away, you can find yourself in a fix.
This is more significant to those with serious assets to invest (>$100K). FDIC insurance only covers $100K at any single bank. There are ways to create multiples of this by opening additional accounts at the same bank, but most people don't properly understand the caveots in this approach and if you think you're not one of them, make sure you ask someone else who really knows...Remember that the FDIC is the final adjudicator in the case that they have to step in, and if they say "uh, no, you don't qualify for X multiples of $100K just because you opened X accounts..." you bite it.
So if you have serious assets, thing VERY hard about whether you want to risk them in a bank that barely deserves to exist, when for a very small hit in interest you can put your money in a bank that is fundamentally sound. (Or spread your money around multiple banks, of course).
Kwad