AI for Scientific Search
arxiv.org
arxiv.org
Anyone know of the best way to do something like:
"Find most relevant papers related to topic XYZ, download them, extract metadata, generate big-picture summary and entity-relationship graph"?
Having a nice workflow for this would be the best thing since sliced bread for hobbyists interested in niche science topics.
Recently found https://minicule.com which is free and lets you search + import, but it focuses more on "concept-extraction" than LLM synthesis/summary.
Disclaimer: I'm the primary author of this project.
It’s not neural network based: it leverages hierarchical mixture models to give a statistical overview of the data. It lets you build these analysis graphs via search or citation networks.
Example: https://platform.sturdystatistics.com/deepdive?search_type=e...
https://platform.sturdystatistics.com/deepdive?fast=1&q=http...
I quote the conclusion of the survey:
---
In conclusion, rapid advancements in artificial intelligence, particularly large language models like OpenAI-o1 and DeepSeek-R1, have demonstrated substantial potential in areas such as logical reasoning and experimental coding. These developments have sparked increasing interest in applying AI to scientific research. However, despite the growing potential of AI in this domain, there is a lack of comprehensive surveys that consolidate current knowledge, hindering further progress. This paper addresses this gap by providing a detailed survey and unified framework for AI4Research. Our contributions include a systematic taxonomy for classifying AI4Research tasks, identification of key research gaps and future directions, and a compilation of open-source resources to support the community. We believe this work will enhance our understanding of AI’s role in research and serve as a catalyst for future advancements in the field.
---
I jumped at this because I'm a mathematician who has been complaining about the lack of effective mathematical search for several years.
Most search will totally fail, because this is made of math symbols. Embedding-based search will give various related things involving, say, integrals and Bessel functions. But then I end up opening Gradshteyn and Ryzhik and trying to find where in this book the relevant terrible integrals appear.
This is a common experience for analytic number theorists. And it's a lousy experience.
AI will have a brand crisis once LLMs get abandoned and researchers need to explain the public that the new AI (not LLM based) is different than the old AI (LLM based) which is different from the old AI (GOFAI)
See, you start making a good point in your rant, but then go too much and stop making sense. LLMs are not going to be abandoned. They've "solved" intent from natural language. They're here to stay.
Of course "AI" will get new things. And architectures might improve. And new things will be discovered and added to the tool box. But having the ability to use natural language as input is so invaluable that there's no way we'll just abandon it...