Hacker News Reader

Comment by uludag | Hacker News Reader

I'd be curious what an LLM would be able to accomplish if it was given all pre-modern texts we have access to. Would it be able to come up with ideas we know as modern (political theories, philosophical ideas, scientific theories, etc.)?

I have a hard time believing that if we fed an LLM all prehistoric speech uttered from humans that no matter what, it would never escape the paradigms of those people. If that is the case, than would relying on LLMs just get us stuck in our own paradigms and prevent true "progress"?

View on HN

LLM are just large language models, they're not capable of thinking or coming up with novel ideas. They could enable people to do so by providing food for brainstorming.

Anytime I see an argument of the form X is just Y an alarm bell goes off in my head. While it is descriptive of the components, these arguments ignore the possibility of emergent complexity. Sometimes things are more than just the sum of their parts.

The sun is just a bunch of hydrogen.

A computer is just a bunch of transistors.

Humans are just a bunch of cells.

An artificial neural network is just a bunch of matrix multiplications.

I personally think LLMs are extremely limited and overhyped. But this form of argument seems incorrect to me since it can be used to argue that LLMs cannot do things we already know they can do.

Probably overhyped from a business and media perspective. From a technological perspective LLMs are the first thing we've ever built that can rocket past a turing test.

LLMs are like mentally challenged autistic human beings. The fact that we even built such a thing is a milestone in humanity.

> "...they're not capable of thinking or coming up with novel ideas."

This claim is unfalsifiable given common definitions of "thinking" and "coming up with novel ideas".

No, it is very much demonstrably false.

Here's a simple program that does not even do LLMs that will trivially enumerate all ideas (broken UTF8 handling omitted for brevity):

  for (var numeral = new BigInteger(0); ; numeral++)
    WriteLine(UTF8.GetString(numeral.ToByteArray());

For any given idea in English LLMs will certainly get it faster.

But it is verifiable. We could quibble over "who judges novelty," but I bet if there were regular examples of it doing so, and there were some community agreement the ideas were indeed suitably novel, we'd pretty quickly shout "existence proof!" and be done.

An LLM could come up with an infinite amount of new theories and ideas, of which >99% would be meaningless, if given enough time and energy.

Human’s ability is in guiding the prompt (of AIs, and the mind) on what is worth knowing and which hypotheses are worth testing.

Choosing an action from the countable infinite number of actions is the general framework of free will.

I think the amazing thing about the LLM is that it's not a random text generator so it's not 99% meaningless.

I give it only 50% meaningless.

With nearly unlimited energy, if you set up an indefinite while-loop asking it to continue to expand and make new theories on the same token vector it would approach >99% meaninglessness.

Yeah that's because the only thing changing is some random seed. It's like looking at f(0) = 1 and f(0.00001) = 1.00001 and saying that the function can't produce anything novel because the answer is always less than 2. Hint: try f(99999).

Of course it's all meaningless because most of the input and output is virtually identical. Vary the input heavily and then you will approach 50%. Of course this is assuming the token vector is truly random in terms of subject matter.

It’s not just a random seed though, this would be attempting to squeeze everything out of the LLM network with all the seeds :)

If you wanted to squeeze everything out of the network it's not just varying the seeds. It's varying the token vector.

Agreed, I doubt the full squeeze would be 50% correct on new hypotheses though. Would be a fun PhD thesis if I went back to grad school :D

I've wondered that too. If we trained an LLM on only the scientific content that predates Einstein could it come up with general relativity?

How much coaxing would it require to get it there? What would that process be like?

Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process?

>> Can we learn from that and get an LLM trained on all scientific material pre Einstein and post to discover new physics stuff from what we learn from that process?

The parts of an LLM that teach it language is not disconnected from the parts that teach it facts. Good luck teaching it language with only pre Einstein data.

Why can't you just omit all modern content related to physics? There's lots of training data that doesn't mention any of Einstein's work.

Maybe you could, but any influence these concepts had on general culture might still leak in.

To me that's an interesting feature, not necessarily a bug.

How much would have to leak in to guide the LLM to the right conclusions? Is it a matter of quantity or quality? Can we successfully eliminate all leakage?

i just read this paper on arxiv and im still trying to wrap my head around the implications so the authors got over 100 nlp researchers to write down novel research ideas and then had them review ideas generated by a large language model (llm) without knowing which ones were human generated and which ones were from the llm and the results are pretty fascinating the llm generated ideas were actually judged to be more novel than the human ones p value < 0.05 but slightly weaker in terms of feasibility i mean whats the point of having a novel idea if its not feasible right but still its pretty cool that the llm can come up with stuff that humans havent thought of before

and the authors are saying that this study highlights some of the open problems in building research agents that can generate novel ideas like the llm was bad at self evaluation it couldnt tell which of its own ideas were good or bad and they also found that the llm generated ideas that were too similar to each other lacking diversity i mean thats not surprising right llms are trained on huge datasets but theyre still just pattern recognition machines they dont really understand the context or the implications of what theyre generating

but heres the thing novelty is hard to judge even for experts i mean how do you even define novelty is it just something that nobody has thought of before or is it something that challenges our current understanding of the world and the authors are proposing a follow up study where they actually have researchers execute these ideas into full projects to see if the novelty and feasibility judgements actually translate into meaningful differences in research outcomes which is a great idea i mean thats the only way we can really know if these llms are useful for accelerating scientific discovery or not

anyway im rambling on now but i just think this is a really interesting area of research and im excited to see where it goes can we really use llms to accelerate scientific discovery and what are the limitations of these models and how can we overcome them etc etc

Your output would benefit from proper punctuation. Are you using some kind of speech-to-text application? Certainly looks like it.

Interesting comment nevertheless, if not somewhat difficult to parse.

It's a brilliant idea, but LLMs are predicated upon the massive data of the internet.

That being said, you could take a pre-2018 dataset and test it for discoveries/insights circa 2019 or 2020 and see how well it performs.

Is this how our simulation started?

Reply on news.ycombinator.com