this is why i always come to hacker news for the expert opinions. thank you for being critical yet optimistic.
Given MJ's extraordinary claims and lack of detail, I thought the GP's response was well-calibrated, especially given MJ's unfortunate choice to lean into vaguely implying this has 'medical' utility, despite providing zero evidence (or even plausible theory) their approach could ever have diagnostic value greater than Butterfly's FDA-approved, handheld, full contact USB pocket scanner which is available now and plugs into a mobile phone. They are using 40 of the exact same transducer chip (designed for full contact use) from 200-400 times farther away. You can use the existing full contact Butterfly scanner today and just move it to 40 different angles. It would take a couple minutes longer, provide vastly greater resolution and is proven to have diagnostic value.
If on a website, rank the results; present the 'how I worked it out' info for the best spotters (and you could interview them). Keep the answers secret for a few weeks, then reveal them in a way that the game is still playable.
It's repeatable, every few months you could interview new experts (or the old ones again), get new models.
Kinda like the critical thinking version of images of a pelican on a bike.
I'm also interested in the broader impact of using LLMs in place of web search for general Q&A when we want 'to know things'. It's pretty clear the way LLMs are being used for knowledge acquisition now is often less accurate while 'feeling' more certain. Even if we set aside explicit hallucinations, I suspect it's still less accurate.
It's not particularly helpful; you could easily have done the 5 minutes of work.
I went back in their comment history before LLMs existed and found comments where they claim to be a doctor and sound like they know what they are taking about. I’m not a doctor but my wife and many of our friends are, so I know what they sound like.
But as far as trust goes, Hacker News has historically been a fairly high trust community. LLMs have the potential to change this dynamic, but I don’t think encouraging people to assume that every post is an LLM is helpful. I don’t think a community with that level of distrust is possible, and at that point we should just all walk away.
It is.