I can't believe you just handed over your phone like that. In a lot of these cases, with wannabe thugs, all of them are scared to throw the first punch. These kids act all tough, but when push comes to shove, none of them are willing to deliver the first blow.
By handing over your phone, you backed yourself in a corner. They didn't even have to throw a punch anymore. Now in order to get your stuff back from them, YOU were the one who would have to resort to violence. They could simply stand back and fend you off, which is a lot easier to do, mentally, than assault.