Depending on how much adrenaline your body can produce him hitting you may only make you angrier and stronger, so I say you're way past due to confront this dude already. You can still be knocked out though so just be smart and time your strikes; let him make the first move then ruin him.
If you know there is going to be a fight never let the other guy throw the first punch, that is just not smart.