As the CTO of a cloud startup, I did exactly that.

Apart from the usual brain-teaser and whiteboard coding, we gave candidates 20 minutes to browse through a small database project in eclipse. Afterwards, I checked whether they could explain the data structure of that database.

The outcome of this test was a much better predictor for competence than any other technical test. I can whole-heartedly recommend it.