Does anyone else think this is innapropriate or just me?

Well this past sunday my little sister had her birthday party, and since our birthdays are only 2 days apart she had her annoying little friends over and I had my boyfriend over, along with one of my guy friends I've practically known since I was born. Well my guy friend (A) brought along one of his friends I had never met before (B). I noticed that B had been staring at me all night while we were all talking on my couch, and he hardly talked the whole night. Anyways, later on we all went out to walmart to browse cell phones because my boyfriend wants a new one and my buddy A is freakishly smart when it comes to anything cellular. So it was getting pretty late and I wanted to leave so me and my boyfriend started to leave and while we were saying our goodbyes B said something smart to me, jokingly, to the affect of "oh why you losers leaving so soon?" And being my classy self I replied "bye A, **** you B" just joking around and B yelled across walmart "you wish". What he said didn't really anger me or anything, but I did a fearful side glance at my boyfriend who gets extremely irritated and jealous easily and he didn't really say anything, because it turns out he liked my guy friends and thought they were pretty cool. But honestly, guys, what if another guy said a remark like that to your girl, especially very loudly in a public place? How would you feel, and do you think its a tad bit innapropriate?