What if the candidate has no work experience to speak from? Do you provide him with math problems or logic questions?
People with hobbies that they have invested lots of time in often understand their hobbies' domain space in a very specific way, and can demonstrate their analytical ability when presented with a problem related to their hobby. An example might be to present a photographer with a challenge one how to get a given picture, or a collector with the challenge of spotting a fake, or a rock climber with a particular problem/route.
I don't think this really provides a foolproof methodology, but when people have strong extra-curricular domain knowledge, I find that they are often able to answer higher level questions about those areas. This is a good proxy for general analytical ability, and helps get past the problem of testing them on something they may know nothing about, which doesn't really highlight analytical ability at all.
The downside to this is that you have to know about their hobby to ask a good question, and you have to put in some effort to ask a good one. This is good, since even casual effort on your part can give a boost to the level of your "test".
Source: I am an Analyst, with lots of hobbies. I have found my own hobbies have prepared me very well for being an analyst that deals with multiple subject matters at a time.
Note: I am referring to analytical abilities as reasoning, judgement, intuition, etc. Not to be confused with skills in Analytics which refer to math/statistics/data science.
Identify the analytical skills that are important in a role. Find parallel scenarios where such skills are also needed that a person most likely to encounter either as hobby or in another familiar personal/professional settings. See how the person approaches the scenarios. After reviewing the resume and information available about candidate before interview, you most likely will be able to come up with certain line of questions that may give insights into specific analytical skills required for the role while not being unfamiliar scenarios to the candidate.
For example, once a candidate during interview mentioned setting up his own home network for Internet connection. After discussing more details about his home network, I just asked him about how he will investigate "Page not found" message in a browser on a PC on his network. The responses and ensuing discussion told me a lot about the candidate capabilities in investigating and resolving problems that was the requirement for the role. Another time, a candidate mentioned liking Chess and American Football. After discussing chess and football with him for while, I just asked him to compare and contrast chess and football. The discussion told me a lot about the candidate capability in finding similarities between two seemingly unrelated situations and able to take lessons from one and apply to other.
Analytical skills are simply the ability to develop new information and insights from a mix of information that is present and identifying and acquiring information that is necessary and not present.
You say "Mary went to the movies, how much did she pay?" Analyst questions is "Is there any thing special about Mary?" Interview question is "What characteristics of Mary do you consider special?"
Now if the interviewee can come back with the typical spread of movie prices (child, adult, student, senior) that is the minimum response. They get more credit if they also include matinee vs prime-time shows, they get still more credit if they test the assumption that Mary was going to a theatre that required payment, and full marks if they want to be sure that Mary does not own, or work at the theatre where the movie is being shown.
Analytical skills are, in my mind, being able to identify all of the variables that might bear on the solution of the problem, ranking them in order of how likely they are to come into play, and constructing alternatives based on the most likely combinations of variables. And doing that quickly and methodically without losing track of the variable expansions in the process.
The reason I prize those skills in an employee is that it gives them the capability to solve poorly specified problem tasks. Flipping my question around, as a manager I would much prefer to tell my employee "Give Mary money for the movie tonight." and be assured that they will expand the variable list, talk to Mary if necessary to fill in missing gaps, and then disburse the funds. But if they either freeze up unable to break down the variables into an action plan, or if they simply provide the "typical" money needed without figuring out if that will be too much or too little, then I know that managing them will take too much of my time. HTH
Despite my objection at the end, I really did like this example as a way to explore a person's ability to ask good questions.
E.g. When hiring a marketing role I'll give someone some campaign materials before the interview (when they arrive so they don't discuss it with friends/family) with instructions to tell me how they would improve this. There's no correct answer but I want to see how they approach and consider possibilities. I'm more interested in the 'why' they say something than the what. Combine this with an Excel or relevant technology test and you should have a good idea of if they can use critical thinking and logic to meet goals. Another question I've used is what business in another industry do they think is looking to change the market. Then challenge their views and see where they go.
For soft skill roles basic logic is seriously under-rated. Also finding someone that can look beyond the obvious. Look for someone that sees the world differently, like a Karl Plinkerton, but with some more common sense of course. I personally don't want hive-mind in a team. I want people that can respectfully disagree at all levels, provide options but then crack on with whatever decision is made. Its surprisingly hard to find this.
I also weed candidates out immediately in the job description by throwing in a sentence in the middle that instructs them to send their resume with the subject line "entropy" or some other key word. When I get emails with resumes I simply delete those that don't have the key word.
For example, people with mild ADHD will probably fair poorly on your tests. Now, if you are hiring for something that requires profound attention to detail (dispensing medicines in a hospital), keep on. If you are trying to measure analytical abilities, you should try measuring that, not some proxy that has little to do with analytical abilities.
(and the reverse also applies - I know people with great attention to detail but that I wouldn't rate particularly highly on analytical skills).
If they have paper experience (or a solid github), pick apart their decisions and lessons from the experience. Look for strong opinions.
If they're "Jr" and have no paper experience, summarize some problem that your company faces every day and ask them how they would solve it.
For Hacker News, that might be how they'd store a comment tree in the database and how they'd render it to the user. Did they forsee any problems? Were they self-aware and recognized what they didn't know?