Throwing "Boy" on anything gives it a demeaning characteristic. Hell literally just referring to someone as "boy" is demeaning as it emasculates the other person and reduces them in stature. Imagine referring to anyone at work, a place where people tend to be more sensitive about social standing, colloquially as "boy" and watch the fireworks goooooo
White Boy is a demeaning phrase used by any number of minority groups to describe a sort of pampered upper class wuss naïve to the ways of the real world. Its certainly stereotypical and can certainly be considered "racist" in the appropriate context.
I guess what complicates these discussions so much is a sort of unspoken understanding of racism being both about otherization *and* power. Since minority groups are unable to have as much real world influence and pull as the majority group, derogatory phrases are generally not identified as such within these communities.
I.E. if laundromats for example are owned proportionally by race, 70% would be owned by Whites and 10% would be owned by blacks. If Whites hold discriminatory attitudes toward blacks, that potentially cuts off 70% of the market for a service to blacks, while if the same were true in reverse, then that cuts off 10% of the market to whites, a clearly disproportionate result. Without the ability to enforce their discrimination proportionately, blacks would be considered "less racist" in this scenario.
On the other hand, the white majority group tends to not see group dynamics (owed in part to being the norm) and values individual dynamics more strongly. As such, you end up having these discussions where one person is arguing from a group dynamic (I can't be racist because we have no power to enforce that racism on whites) while the other person tends to argue from the individual dynamic (That is racist because you said a thing that delineates on race, it is ideologically racist and your ability to enforce that upon me is secondary to that fact).
And as such we get these page long discussions where everyone talks past everyone else and gets nowhere.