There would be, certainly. We don't have to eliminate them, though. Just modify them so they can't carry the diseases anymore. Both are options that we could carry out right now, if there weren't checks and balances in place to prevent it. Which admittedly is probably a good thing, since eliminating them would be the easiest thing to do.
I am all for this type of intervention. I do think we should be careful, but this isn't much of a slippery slope. Mosquitoes in particular cause immense human suffering. We've already done untold damage to our ecosystems, why stop now. Nobody would miss the mosquito even if we did decide to wipe it out, and the few species specifically dependent on them would just be the casualties of war I guess.