Tell "your team" to come up with a better reply, since your statement means "wrestling" IS a "sport" 
Ehm that's a weird reversal??? Do you really think your statement follows from that?
More importantly, you DO play with another guys balls in wrestling ... 
No more weird than saying ball-related games aren't "much of a sport". Soccer, something the majority of Europe follows and have KILLED people about, is most definitely a sport, something I can freely admit even if I don't care a whit about watching, playing or going to see - ever.
Football - for the guy who believes racing an automobile IS a sport, and football wouldn't be much of a sport, take the two athletes and switch them around. I'd be willing to bet that they'd BOTH agree the others' sport is tougher than they thought, and at times, pretty damn frightening.
The idea that you play with someone's balls while wrestling is simply idiotic. Either the "fake" wrestling, standard high-school/college type of wrestling, Greco-Roman, whatever - if you consider that "playing with another guy's balls", you've put as much effort into following what that sport is about (and knowledge of it) as I've put into soccer (as well as auto racing of any kind).
Lastly, while I can't ever fathom watching auto racing of any kind, I'm not about to make a stupid statement like "it's not much of a sport". I could have said something as equally mindless about auto racing such as "since my grandma is able to drive around in a car, that doesn't make auto racing much of a sport". Yeah, I pointed out wrestling, because you DON'T use a ball, you DON'T "play around with another guy's balls", and it IS a sport. Follows perfectly since I chose to actually use an example instead of "that's not a sport, and it's because of some (half)witty rejoinder I never put much thought into".