Some vegetarians are just very grossed out by it (I'm one of them).
I generally look the other way at very minor cross contamination, its probably going to happen once in a while, especially if I don't know about it, but if I taste meat juices in my food I completely lose my appetite. I once bit into sausage that found its way into my pizza (I suspect it fell into the cheese) and I couldn't eat pizza for a while after that. (of course, for a burger that's supposed to taste exactly like beef, you wouldn't know...)
If you think that makes me crazy or whatever, then so be it.