A particular complement might be sexual, but we shouldn't assume that. For all we know, the guy is gay, and we are now discriminating against that.
Furthermore, commenting directly on the awesomeness of someone else's body is also not inherently sexual. Have you never admired a female athlete, the way you admire a male athelete?