#8 measures self-awareness. Many people can answer this question with "I know I could be much better at X and I am working on it" which is what it's about.
According to recruiter and many answer here it appears people are spending days at job throwing numbers to the wall.
Can anyone explain what these numbers with massive uncertainty are used for and why go tough all that effort if you still have to go back to sources and references to fix up the estimate anyway?
Eg. if I guess NYC population is 10M, with every 100th household having a piano, I can guess there are 100k pianos in NYC. If each of them needs tuning once every two years, that's 50k tunes each year. That's roughly 1k tunes each week, which, if a tuner can do 2 a day (5 work days per week), is 100 tuners.
And then it'd be worth comparing to tuning of venue pianos which are tuned more frequently: let's say 10k venues with pianos requiring tuning every month, or 2k tunes a week. So another 200 tuners.
You can also talk about pattern of work instead of market demands (eg most tuners only get 1 tune a week, so we can multiply the number of tuners by 10), just to show off you are ignoring other variables knowingly.
Obviously, each of the guesses has a large error margin, and errors multiply, so you can be off by way more than a single order of magnitude.
But just coming up with things to include in your calculation, and not being scared of exploring such random situations with little data you know, signals some generic problem solving ability.
Eg for a better answer, I now know I need exact number of people and venues in NYC, I need to ask tuners how often are pianos tuned, get some data on how frequent are pianos in homes, how long does a tune take, etc.
While I think I am decent at these questions, and I do believe they signal a very generic problem solving ability, I agree they are terrible interview questions for most software engineering roles, since you rarely hire for such generic ability, and might even get more out of a more specific question/test..
It's like the Drake equation - it doesn't matter what factors you choose (you can always come up with more) or what values you assign them (you can't know what most of them would be anyway). It's meaningless as a whole. And also irrelevant to software development or most anything else; at best it would be only very loosely relevant to managing a piano tuning company.
I have no idea what those values are! I am still able to make guesses to show that I can figure out a bunch of variables that affect the final value. And since I am multiplying data that's off maybe even by few orders of magnitude, errors multiply as well: I am aware of how erroneous input data affects the results too.
Is it relevant to software development as a whole? It definitely is.
You've got to provision web workers for new service being developed: you can only (guess)timate number of expected users, you can guess how much CPU time each request will take, figure out cost per unit, look at bandwidth requirements, and make your pick. To improve on your guess, you actually go and measure request times (p50, p99), decide on the number of users you can sustain with your team cost effectively, measure network usage, etc.
Or you need to decide on what type of DB to use? Depending on access patterns (read heavy, write heavy, time series...), data size and scalability demands, team familiarity with different tech, you make a guess at what should work: you can only measure after you've got the system exposed to real, live load.
Now, I still think it's a bad interview question: people can be perfectly comfortable answering above questions, yet not see that they are in abstract exactly the same as the number of piano tuners in NYC. It tests for that generic and general ability to estimate anything, and to break down any problem.
Yet some people get flabbergasted (like you seem to do?), since they think too much about specifics of each of the numbers they use and how wrong they might be. Basically, they have a hard time of letting go of the inaccurracies :)
As such, it's a mixture of tests: supposedly only tests you problem breakdown skills, but in reality tests your confidence in using inaccurrate numbers or lack of problem domain knowledge to solve a problem.
Which makes it a bad interview question unless both of those is what you want to learn about someone.
It's similar to estimating novel work.
It shows that one can break a problem down into more manageable pieces. Being more manageable means you can get a better estimate by delving deeper.
The question is not about finding the right answer, but about employing the right methodology.
And if you've got next-in-the-chain taking estimates as hard boundaries or facts, you need to be explicit they are not, and hopefully teach them to be a bit more suspicious of everything.
Even then, it's ok if someone else takes your ballpark and runs with, as long as you are willing to take responsibility. If you are higher in job title, be explicit this is what you are asking them to do, and that you will take the responsibility! Invite them to improve upon it as they learn more, though!
Ability to BS and multiply, more or less. Feels like a waste of a question even for the one asking it!
> Yet some people get flabbergasted . . . since they think too much about specifics of each of the numbers they use and how wrong they might be. Basically, they have a hard time of letting go of the inaccuracies :)
Having a hard time letting go of inaccuracies seems justified if you're in a more technical profession. Too often there's good engineering reasons for not wanting to excuse loose reasoning with bad guesses. On the software dev side, there are also too many cases where those kinds of fudged numbers are inevitably used against us.
I've also seen people fail #8 in interviews for junior positions. They can't introspect. It sounds impossible but it happens.