I totally agree with this. The best interviews I've had with candidates have gone like this, and not only do you get a feel for their knowledge in a subject in a more relaxed way it also gives you a bit of a feeling if this person would fit in the team or not.
Being on the other side of the table, interviews done this way has landed me in my most interesting/best jobs, and in the best teams, in my opinion.