Well in the old days people would use ladders and climb into the plane. As for now, they could maybe have one entrance that is at the plane level and the rest of the entrances be on the ground. If you want to get to your seat faster and get onto the plane sooner, you walk outside and climb into the plane. The skywalk that goes directly into the plane would be for those who are willing to wait or for the disabled.
You'd end up with somebody in the back that wants to get out the front because they don't want to go down on the tarmac. People trying to go opposite directions will make it worse, not better. You can tell them they need to wait all you want but it won't work, they'll still stand in the aisles and block everyone else. Also, having two exits at the front and two at the back won't be any better than one at each end. The bottleneck on single aisle planes is getting down the aisle, adding an extra exit once they've gotten out of the aisle isn't going to speed anything up.
Unloading people from the rear of the plane onto the tarmac is probably a bad idea anyways. With the service trucks, the engines that may be turning on and off, and just general safety and security it would be ill advised to have passengers wandering around on the tarmac. It would require a person or two back there to herd the passengers. Paying extra people for that could very easily wipe out the savings you'd get for unloading a few minutes faster.
just went to barbados a few weeks ago and at the barbados airport they unload the airplanes from the front and rear exits
I've been in one airport where they did they unloaded from the front and back, but I've never seen one where they load from both ends.