You should probably not be very honest. Any criticism will likely on put you in a negative light, and you might want to have some references from that company at a later point in time.
"You know, now that you ask, it's wonderful here and everyone is outstanding. To tell you the truth, I'm not sure I've made the right decision."