Physical Touch

Hey, I'm curious on what everyone things about this situation I went throught. I'm a 24 y/o straight male and thing I'm sending some signal to one of my friends. I grew up in a family where physical contact between two people even of the same gender was normal. As long as it did not occur below the waist of above knee. Such things as hugging, pats on the back, arms on each other's shoulder. It never was a big deal. Well, the other day my friend Brad and I were visiting out friend Sam at a campground. We had to cram in the back of our friends car and she had her arm on the seat behind my back and joked about it. So me, being funny decided to put my arm behind Brad's back and he went nuts saying "Get your had off my back" which was weird because he is never stern like this about anything and all I meant was joking.

I do notice this in other things like he is paranoid about other people touching him, sometimes even girls. Did I do something wrong? Is there a reason he rejects physical touch. I mean I could totally understand if I had grabbed his ass or balls but I didn't. All I did was put my arm around his shoulders like many friends (even guys) will due. Even in high school I did this with my guy friends and we would joke about it. It's like a straight way of showing physical emotion and is not suppose to be taken sexually. Can anyone shine some light on the subject for me. I did email him and aplogize for freaking him out. But I still don't see why he would freak out like that. Thanks in advance for any help you all can provide.