> As always with any LLM you should double-check its final, specific answers. It does occasionally hallucinate when information simply isn't available.
It also sometimes completely botches it when information is available. For example a while back someone cited Musk only scoring 730 on the math SAT as evidence that there is something from with the test.
I looked up Musk's age to figure out about when he would have taken the SAT then asked Perplexity what percentile a 760 would have been then. It gave me an answer that as far as I can tell was right (~90th).
I then wondered what my percentile was, so asked it what percentile 790 would have been when I took it. It told me it would have been 17.something, where that something had 5 digits.
That was obviously completely wrong because (1) there is no possible way it could have data that would justify giving an answer with 5 digits after the decimal point, and (2) the maximum possible score was 800 and scores were a multiple of 10, so for 790 to have been 17th percentile would mean that 83% of people who took the test scored a perfect 800.
I told it that this was clearly absurd.
It responded that I was completely right and said it was going to try again. On the retry it gave a reasonable answer that I knew from what I remembered was in the right ballpark and not given to ridiculous accuracy.