Actually, I think this is a fair way to evaluate new hires. That said, I hope you don't look over their shoulders? I can't tie my shoe laces with someone looking over my shoulders. Or, secretly view their progress? I'm o.k. With reviewing their progress at the end of the day though?
Plus, if you are going to hire in this fashion, why even bother with resumes, or extensive resumes? Just bring people in for a test? If they do well--hire them?
I always felt a test is the fairest way to hire new employees. Tests do away with upperclass privilege(better contacts, padded resumes, etc.) I wish more companies vetted employees in this fashion. It might get rid of something I have always dreaded, and get slightly nauseous when I see the behavior in action--and that is Networking. Sorry, I always thought it was phoney.
I still imagine your company relies heavily on resumes, and uses the test--because they can? Meaning--there's still a lot of desperate software engineers out there; willing to do anything to land a job?