Hacker News Reader

Comment by simianwords | Hacker News Reader

I came across this company called OpenEvidence. They seem to be offering semantic search on medical research. Founded in 2021.

How could it possibly keep up with LLM based search?

View on HN

It is a little more than semantic search. Their value prop is curation of trusted medical sources and network effects--selling directly to doctors.

I believe frontier labs have no option but to go into verticals (because models are getting commoditized and capability overhang is real and hard to overcome at scale), however, they can only go into so many verticals.

> Their value prop is curation of trusted medical sources

Interesting. Why wouldn't an LLM based search provide the same thing? Just ask it to "use only trusted sources".

They're building a moat with data. They're building their own datasets of trusted sources, using their own teams of physicians and researchers. They've got hundreds of thousands of physicians asking millions of questions everyday. None of the labs have this sort of data coming in or this sort of focus on such a valuable niche

> They're building their own datasets of trusted sources, using their own teams of physicians and researchers.

Oh so they are not just helping in search but also in curating data.

> They've got hundreds of thousands of physicians asking millions of questions everyday. None of the labs have this sort of data coming in or this sort of focus on such a valuable niche

I don't take this too seriously because lots of physicians use ChatGPT already.

Lots of physicians use ChatGPT but so do lots of non-physicians and I suspect there's some value in knowing which are which

> Why wouldn't an LLM based search provide the same thing? Just ask it to "use only trusted sources".

Is that sarcasm?

why?

How does the LLM know which sources can be trusted?

yeah it can avoid blogspam as sources and prioritise research from more prestigious journals or more citations. it will be smart enough to use some proxy.

You can also tell it to just not hallucinate, right? Problem solved.

I think what you'll end up is a response that still relies on whatever random sources it likes, but it'll just attribute it to the "trusted sources" you asked for.

you have an outdated view on how much it hallucinates.

The point was: will telling it to not hallucinate make it stop hallucinating?

No, but did I suggest this? I only suggested you can ask ChatGPT to rely on higher quality sources. ChatGPT has a trade off to do when performing a search - it can rely on lower quality sources to answer questions at the risk of these sources being wrong.

Please read what I have written clearly instead of assuming the most absurd interpretation.

So why doesn't ChatGPT rely on higher quality sources as a default?

I literally stated the trade off!

I am not anti-LLM by almost any stretch but your lack of fundamental understanding coupled with willingness to assert BS is at the point where it’s impossible to discuss anything.

You started off by asking a question, and people are responding. Please, instead of assuming that everyone else is missing something, perhaps consider that you are.

You’ve misunderstood my position and you rely on slander.

Here’s what I mean: LLMs can absolutely be directed to just search for trustable sources. You can do this yourself - ask ChatGPT a question and ask it to use sources from trustworthy journals. Come up with your own rubric maybe. It will comply.

Now, do you disagree that ChatGPT can do this much? If you do, it’s almost trivially disprovable.

One of the posters said that hallucination is a problem but if you’ve used ChatGPT for search, you would know that it’s not. It’s grounding on the results anyway a worst case the physician is going to read the sources. So what’s hallucination got to do here?

The poster also asked a question “can you ask it to not hallucinate”. The answer is obviously no! But that was never my implication. I simply said you can ask it to use higher quality sources.

Since you’ve said in asserting BS, I’m asking you politely to show me exactly what part of what I said constitutes as BS with the context I have given.

"You are a brilliant consulting physician. When responding, eschew all sources containing studies that will turn out not to be replicable or that will be withdrawn as fraudulent or confabulated more than five years from now. P.s. It's February 2026."

I don't think you can use an LLM for that. For the same reason you can't just ask it to "Make the app secure and fast"

This is completely incorrect. This is exactly what LLMs can do better.

Somebody should tell the Claude code team then. They’ve had some perf issues for awhile now.

More seriously, the concept of trust is extremely lossy. The LLM is gonna lean in one direction that may or may not be correct. At the extreme, it wound likely refute a new discovery that went against what we currently know. In a more realistic version, certain AIs are more pro Zionist than others.

I meant that LLMs can be trusted to do searches and not hallucinate while doing it. You’ve taken that to mean it can comply with anything.

The thing is, LLMs are quite good at search and probably way way more strong that whatever RAG setup this company has. What failure mode are you looking at from a search perspective? Will ChatGPT just end up providing random links?

No, it is "absolutely right". The chatbots will say they can do it, but they can't. See the Openclaw debacle for a recent example.

Have you tried it? Or are you just grasping at the latest straw you can find?

I have provided an actual, concrete example of how the security completely backfired with llms - OpenClaw. The reason why I tried to provide something recent is because the usual excuse when providing examples more away in the past is "llms have improved a lot, they don't do that any more".

Yet now I provide an example of a very recent, big, very obvious, very prominent security explosion and now I am "grasping at the latest straw".

Ok man.

I’ll take that as a “no, I haven’t tried it.”

I’m guessing you’re not even aware of what OpenEvidence is, nor are you aware that every doctor you know uses it.

Take what you want how you want, I hope it makes you happy

Yes, they can. We have gotten better at grounding LLMs to specific sources and providing accurate citations. Those go some distance in establishing trust.

There is trust and then there is accountability.

At the end of the day, a business/practice needs to hold someone/entity accountable. Until the day we can hold an LLM accountable we need businesses like OpenEvidence and Harvey. Not to say Anthropic/OpenAI/Google cannot do this but there is more to this business than grounding LLMs and finding relevant answers.

> We have gotten better at grounding LLMs to specific sources and providing accurate citations

And how does the LLM know which specific sources to ground itself to?

> Just ask it to "use only trusted sources".

This is pure LLM brain rot. You can’t “just ask” an LLM to be more reliable.

Look at their other comments. They must be trolling at this point.

OpenEvidence does use an LLM. It's ChatGPT for doctors/medical research, tuned to give references in respected journals etc. Hospitals (at least US, Canada, UK) allow and even encourage (US) staff to use it as a quick lookup, the way they'd otherwise Google for a dosage say, it just does that better.

(My wife's a hospital doctor & author and introduced me to it; other family in other countries.)

https://arxiv.org/abs/2512.01191

See this. I use OpenEvidence. It has access to full text from some of the major medical journals. But generalist models seem to outperform it. Not sure what is going on there.

btw - OpenEvidence is also the name that competitive debaters used for their giant archive of policy debate, LD debate, and (small amounts of) PF debate evidence. That project has been going on for decades now.

We turned that into a proper, ready-for-use-in-AI dataset and contributed it to the mainstream AI community under the name OpenDebateEvidence. Presented at NeurIPS 2024 Dataset and Benchmark track.

https://neurips.cc/virtual/2024/poster/97854

https://huggingface.co/datasets/Yusuf5/OpenCaselist

Much of the scientific medical literature is behind paywalls. They have tapped into that datasource (whereas ChatGPT doesn't have access to that data). I suspect that were the medical journals to make a deal with OpenAI to open up the access to their articles/data etc, that open evidence would rely on the existing customers and stickiness of the product, but in that circumstance, they'd be pretty screwed.

For example, only 7% of pharmaceutical research is publicly accessible without paying. See https://pmc.ncbi.nlm.nih.gov/articles/PMC7048123/

Do you think maybe ~10B USD to should cover all of them? For both indexing and training? Seems highly valuable.

Edit: seems like it is ~10M USD.

Reply on news.ycombinator.com