My initial worry is the subjectiveness of "offence".
As a speaker I like to think I'm putting on a show, I try to make my talks entertaining by throwing in a few jokes. If jokes were discouraged at a conference, I'd neither want to speak or attend. Unfortunately, I don't think it's possible to crack a joke without potentially causing someone offence. Maybe my favourite chicken got run over trying to get to the other side.
An example I trot out when talking about "offence" happened at uni. We were watching a comedian, he was making jokes about pedophilia and rape much to the delight of the audience and specifically one of my flatmates. I thought the comedian was terrible, not because of his subject matter (I don't believe any subject is out-of-bounds for comedy), I just felt his jokes were lazy. He then made a joke about bulimia, my flatmate slammed his drink down in anger. I asked what was up, he replied "My girlfriend suffered from bulimia, I've seen what it does, I don't think you should joke about it". I was astounded by the hypocrisy, and replied "I agree, people who think they can joke about bulimia make me sick".
My point is, it's impossible to predict what may offend a given audience, and I don't think people should try not to offend anyone. My own humour is pretty childish and has offended people, in one talk I refer to this image http://wtfhub.com/wp-content/uploads/2010/12/comic-captain-a... and later describe the action of a particular browser as "wanking us in the face", making it clear that I'm using "wank" in the comic onomatopoeic sense rather than referring to a sexual act. This joke has offended a small number of people, causing one to walk out. I don't know if they didn't get the joke, or simply find the sound of the word offensive, but I have no intention of removing the joke when I give the talk again because the vast majority of the audience enjoyed it. This isn't a male/female thing either, and I worry about the suggestion that anything vaguely sexual is offensive totally and exclusively to women.
Anyway, I'm getting off topic. I think my point is we need to quantify what "offensive" means before it becomes a solid rule. Or make the rule as subjective as the word itself, eg "If you find something offensive, let us know and we'll investigate it".