As one of OP's direct reports, I can attest to this system working really well and, at least for me, being a much faster and more comfortable experience than doing a written peer review.
Why can't employees give feedback directly to each other instead of having to go through the manager? Direct feedback might save quite some time. Thoughts?
That would be ideal, but a lot of people are uncomfortable doing that, and you would not get critical feedback as often or as reliably.