Yes, the stated question is about one specific individual. But unlike most of the answers posted so far, "RcouF1uZ4gsC"'s answer acknowledges that there are people for whom, yes, it might be a red flag. As opposed to your answer above, which states "strange enough to think that something like this matters," implying there is absolutely nobody that might raise a concern.
As I mentioned, if the interviewer never heard of Linus Torvalds or Bill Gates, would you really say "it doesn't matter?" If yes, then I kind of wonder about your judgment a little bit. If no, then the question could be re-worded as, "Is Joel Spolsky sufficiently famous or influential among professional software engineers that you'd hesitate before working for someone who had never heard of him," which is a more nuanced question, as it asks the respondent to place him along some spectrum and to determine where the cutoff point is.