Depends what you're doing. All phones can lag, think about it, they really are just small computers. So you have to take in to account all of the same things you would on a computer. What hardware, firmware and software is in use, the resources utilized at any given moment, the tasks being performed, etc.
I had the S2, Note, S4 and now the OnePlus. ALL of them lagged. The OnePlus now that I installed OxygenOS Lollipop is by far the fastest and smoothest, but even the Note had very good hardware at the time, the issue was the version of Android at the time + the Touchwiz UI that was horribly unoptimized.
Before the Lollipop upgrade on the OnePlus, many things were smooth, others lagged a lot. Hangouts lagged a lot for sure. Now on Lollipop, Hangouts is buttery smooth with very little exceptions. Much faster.
Take control of what's happening on your device. Root it, remove unnecessary clutter, use something like App Ops to control what permissions apps are actually using (you'll discover in the process that many apps are using things like Location even when you "disabled' it from the regular menu, accessing the camera when you're not using the app, etc - these things all use precious resources and battery).
If you're using a phone that has a laggy overlay like Samsung's Touchwiz, look for a good custom ROM such as Cyanogenmod or Paranoid Android, XDA is a good source for that stuff.
And finally, don't use the lowest end devices. The flagship devices are always going to be faster. Flagship Android devices perform similarly to the Apple offerings. Until now, Apple has always had faster camera performance than any Android phones, but maybe they finally bridged the gap there, not sure.