It does have one positive side effect, however: it cleans out your sinuses like nothing else I've seen. :)
A large part of the effectiveness is also psychological. When people first experience CS they tend to overreact and experience secondary symptoms not created by the CS. After a couple exposures, you know what to expect and CS becomes more of a nuisance than something to fear. You don't intentionally walk into a cloud of CS but you can kind of shrug it off if you do.
A major point of the exercise is to instill trust in your gas mask and your ability to use it correctly.
When I did it, we put on our gas masks, entered a little building full of gas, stood around a bit feeling the tingle on our exposed skin, then one-by-one were told to take off the mask and try to say something (I think it was our name and rank), then run out.