Assuming Microsoft actually measures this stuff, that's kind of surprising to me. I've never placed much weight on being able to write a program on the spot.
I don't disagree, and personally I hate in person coding exercises, but if you keep the question simple and allow the candidate to code in their native language, something fizz-buzz like will go a long way in weeding out the people who have languages on their resume but can't code.
At our company we offer two simple coderpad tasks to be done in advance with no supervision or pressure - write a linked list in CPP and solve a simple puzzle in Python. You pay attention to the candidates who use templates, destructors, smart pointers, even if they make mistakes those are the ones you probably want to hire.
Maybe I don't know enough C++ to understand this, but I don't understand why using fancier features would make you like the candidate more when they are making mistakes. Unless you ask them to write a templated linked list, why would you grade them for automatically deciding to write one (and then making a mistake while doing so)?
Trying to understand the thought process here since I have been on the receiving side of some of these "Look I used the cool feature!" candidate programs in Python, and they often make pretty bad mistakes while using said advanced features. That personally marks them down slightly in my estimation, but curious to know your view is the opposite.
Meta: Maybe the problem with hiring via takehomes/code samples is precisely exemplified by the above discussion.
It's not about cool features. It's about using the correct modern features which make the code safer and easier to follow.
Templates are a relatively advanced feature but trivial to write for such a case and I'm looking for a candidate who understands containers are typically generic and knows how to add the two lines. I take it as a marker of experience.
There's no excuse other than inexperience or old fashionedness to not use smart pointers.
Destructors are also a sort of minimal knowledge because they show that a candidate thinks correctly about memory management.
There isn't really a score to what we do. And in all honesty the first few times we gave the test I didn't really know what to look for, this is my first time from the other side. But it isn't too much to expect a competent cpp programmer to know these things and I'm looking for candidates who are willing to use the open ended opportunity to flex their muscles, provided their code is well structured and doesn't have severe, obvious bugs. I don't claim that this is perfect but it seems suited to screen for the basic kind of knowledge we look for. These are not entry level positions either.
The best part about it? I was already a full time employee at Microsoft. I just wanted to switch teams. I still had to do a full interview loop with the team and well... I didn't get the job.
In terms of people I've hired, if they can solve and code things in real time on a white board, then they can definitely do even better with more time and research.
Everyone does better with research, trial, and error. That's not unique. What's unique is problem solving on your feet in a foreign environment. I want those people.
Better? Sure. Not everyone can go from 0 to competent in a new area within a reasonable time frame.