There are two possible answers here.
One is that sure, a VPE/CTO who cannot evaluate technical contributions in a vacuum is unfit. I agree.
However, a VPE who can't evaluate any specific technical contributions in their organization may not be failing, and in fact probably isn't.
To evaluate someone's technical contributions, there are at least two things you want to look at (there may be more, but there are at least two): resultant impact, and inherent complexity[1] of the problem. A VPE should almost always be able to evaluate the first. They're the one setting goals. They can evaluate a person's contributions on whether or not those helped meet or met those goals.
They won't however be able to know the inherent problem space complexity. That requires a level of in-the-weeds familiarity with the space that you don't get as a non-individual contributor. It's practically impossible to accurately judge how difficult a problem is to solve without either having attempted to solve it yourself, or seeing how many people perform solving it, and most of the time, you don't have many people attempt to solve the same business problem.
You also don't want to solely trust the people working on the project, they have reasons to inflate the complexity, both conscious and unconscious. So you use a lot of things to gain partial signal:
- what they say: "this was hard"
- what other people who had views of their actual contributions say: "their problem space looked difficult and I couldn't find any better solutions"
- what people with historic knowledge of the component/space say: "Their solution managed to clean up years of technical debt that I accrued when designing this system"
- Smaller impact metrics: developer velocity improved when making additional changes to the new system, etc.
But its often not possible for a VPE to directly assess those things, they don't have the context to do that for every employee.
[1]: Note that this is complexity of the problem space and not complexity of the solution. Simple solutions to hard problems are good things, complex solutions to complex problems are acceptable, and complicated solutions to simple problems should be a negative. And yes, you need to account for problem space complexity, otherwise you encourage infighting since performance = impact, then higher impact per unit difficulty will result in better performance per unit time.