I'm colorblind and don't get offended by people asking "is this shirt red or green" when they find out. I started enjoying it once I had enough comeback lines to pick from.
That sounds like a really great way to handle it. Find and learn the best comebacks possibly and make everyone feel good about the situation while showing off your positive attitude and wit.