Yellow and red cards may achieve segregation, but they'll also introduce fear into the mix. A man who receives one of these cards, for whatever reason, will think twice before he talks to a girl at a conference again, just as a girl who has been grabbed once at Defcon thinks twice before ever going to Defcon again.
People deal with fear and stress in funny ways. Adding one more layer of fear in a society that is already a little too afraid of the opposite sex will only backfire more in some other way.
Men who grab women do so because they're afraid to talk to them. Similarly, every time a guy says "hello" to an overly paranoid girl, she'll think that he wants to get in her pants.