In practice, though, since hiring decisions are so subjective, couldn't one justify it with "we felt that they wouldn't be a good fit" and be off the hook?
Another thing to consider: people who file discrimination lawsuits usually do not prevail on the discrimination claim, they prevail on the retaliation claim. It is often quite easy to show retaliation, especially if it is an employee and not an applicant.
I would argue that you can not infer much about someone's job performance based solely on their age.