Because that's not a warning sign at all! boggles
Most people of course answer this question in the same way an interviewee does when asked 'Why did you leave your last position?'. Nobody will ever answer 'that jerk across the hall, I can't STAND that guy'. They will answer something that gives some insight into the company culture. I don't think it's a coincidence, for instance, that easily half the people at Google I've put this question to have answered something along the lines of 'management chaos/confusion', 'the tendency for two internal projects to be competing with each other without even knowing about it', etc.
I should have added the followup to my first question is what they like most. Both can be very informative, of course, and following up with that helps put them at ease for the rest of the interview, since many interviewers aren't expecting a question like this one. But it's really worth it. An interview isn't just to answer 'will you hire me', it also must answer 'will I enjoy working here'. The second is much more important to me.