Hacker News Reader

Comment by jonwinstanley | Hacker News Reader

I wouldn’t expect an LLM to get this right unless it had been trained on a solution.

Am I wrong to think that? Are LLMs in the future going to be able to “think through” actual logic problems?

This is the whole point of the breakthrough related to the emergence of cognitive capabilities of LLMs. They are literally Markov chains. No one expected it to happen to this degree, but here we are.

Almost kind of proves ideas shouldn’t be copyrightable.

Ideas aren't copyrightable.

maybe they meant idea like when you write a book you are transcribing a series of ideas you had

In my mind I was thinking about recipes and code, which are really little more than ideas. My point was that, if such things emerge with a complete lack of creativity, perhaps they don’t warrant protection.

Disney: "Let's agree to disagree."

People say that "they are literally Markov chains", but anyone who has looked at the code for LLMs knows that they are more complicated than that. I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple II after reading about the famous Mark V. Shaney hoax (https://en.wikipedia.org/wiki/Mark_V._Shaney). No neural nets or fancy GPUs required. It's one thing to stress that LLMs aren't magical or self-aware, but the fact is they are way more complicated than simple Markov chains.

> People say that "they are literally Markov chains", but anyone who has looked at the code for LLMs knows that they are more complicated than that.

They are literally Markov chains according to the mathematical definition. The code is complicated. Having complicated code doesn't mean it's not literally a Markov chain.

> I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple II after reading about the famous Mark V. Shaney hoax (https://en.wikipedia.org/wiki/Mark_V._Shaney). No neural nets or fancy GPUs required.

I don't doubt this. You can make a Markov chain by just counting the frequency of letters that follow each letter giving one that has a context window of one or two characters. That is a very simple Markov chain. You can make it by hand. You can make ones with more context window like a dozen characters or a few words, using sophisticated smoothing and regularization methods and not just frequency counts. Those are also simple Markov chains that you can do without neural net or GPU. Then you can also make a Markov chain that has a context window of thousands of tokens that is made from neural nets and massive training data and differentiable tensor computing libraries with data centers full of hardware linear algebra accelerators. Those are some even bigger Markov chains!

> LLMs are way more complicated than simple Markov chains.

That's true, they are more complicated than simple Markov chains, if by simple Markov chains you mean ones with small context window. LLMs are Markov chains with large context window!

How big is the state space of the Markov chain corresponding to a LLM generating a sequence of tokens? Wouldn't it be (size of the vocabulary)^(size of the context window), i.e. ~ (100k)^(4k)? How useful is it to conceptualize LLMs as Markov chains at that point? For example, is there a result about Markov chains with interesting implications for LLMs?

A Markov chain with a large context is still literally a Markov chain.

Maybe you are used to Markov chains being shitty at language so you are confused how an LLM can be a Markov chain even though it's good at language and has some amazing emergent cognitive capabilities. That's a problem with your conception of Markov chains, it's not an argument that LLMs aren't Markov chains.

Finally, a Markov chain with a context space that cannot be practically iterated over (e.g. all possible 10k token contexts) can still be useful in ways that are shared with smaller Markov chains, even though if this weren't true it would still be a Markov chain. For example you can greedily generate tokens from it, calculate likelihoods, do some beam search, select multiple choice tokens, etc.

I don't think so. Consider the following conversation from UndiMix-v2-13B (a local LLM running on my Mac M1 via llama.cpp). It gets it wrong at first, but then gets it right with a little hint. I don't see how the hint could work if it was merely repeating an example it knew.

User: Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?

Llama: Hello there! I'd be happy to help you with that question. Let me think... If each of Sally's brothers has two sisters, then together they would have 6 sisters in total (2 x 3 = 6). So, Sally would have 6 sisters as well. Is that the answer you were looking for?

User: no. Think about it. Sally is a sister to the brothers

Llama: Ah, I see what you mean now! In this case, since Sally is one of the siblings, she only counts as one of her own sisters. Therefore, Sally has only one sister - herself. Does that make sense?

User: yes

Llama: Great! I hope that clears up any confusion. Please let me know if you have any other questions or need further assistance.

I don’t think ‘Sally has only one sister - herself’ is the right answer.

The initial response had 2 problems - not counting Sally and counting sisters separately for each brother. Your hint pointed out the 1st problem, but then LLM should still provide wrong solution. This makes me think that maybe it would try another answer and explanation no matter what you said. If your reply was just "no, think again" it could also provide the right answer

She counts as her own sister :D I can be sister of myself?

Llama: ... Therefore, Sally has only one sister - herself. Does that make sense?

User: yes

----

Um... no!

I see no reason why not. You would need to represent dyadic relationships and whether they are reciprocal, transitive, etc., weigh probabilities about the intended meaning (LLM magic already), then use sentential logic rules, right?

I've been talking to GPT and asking it long multi part questions and it does seem to 'think through' them, and able to provide negative responses.

Specific logic questions like in this post, or long math formulas seem to still be a struggle. But I get the impression it is just a matter of time before it is scaled up to handle them.

You can argue whether it is 'reasoning' about them or not. But if it becomes indistinguishable from a human, then does the word 'reason' even really apply to the human anymore? Both are black boxes, giving answers.

Indeed, anyone who says llm's cannot reason since it is just reformulating things it remembers, is unable to explain what reasoning is, let alone how it would not be reasoning.

In the end, if the result is indistinguishable from a treat people call reasoning, let's call the behavior reasoning.

You could achieve thinking though logical problems by adding chain of thought or tree of thought. Then the LLM will try to break stuff down into logic it’s already seen.

Reply on news.ycombinator.com