Yes - and be very direct about this. Ask them "So why do you like this job? Why is this a great place to work?" I've had someone pause, look at me and tell me why it was a bad place and I should run away.
I sent him an explicit "thanks for the generous offer, but I've chosen to go elsewhere after consideration" email so he had something to forward, thanked him kindly privately, and ran like hell.
It's pretty amazing what you'll run into across a career. I've probably seen more WTFs from the hiring side of the table than from the people I've interviewed to hire.