LLMs Are Interpretable | Hacker News Reader

LLMs Are Interpretable

timkellogg.me

17 pointstkellogg10 comments

I think interpretable is a overloaded term. If you use RAG, sure you can look at the retrieved text and understand what is being used, but at that point you're just interpreting the retriever.

LLMs need to be interpreted so that they can be edited have their biases understood in a systematic way. However just as people aren't "interpretable" these algorithms are not going to be able to display their inner workings with 100% confidence. It's going to remain probabilistic, which might be fine for the majority of use cases. I think we're coming from an age where everything was 100% interpretable because we knew what was going on inside the machine (e.g. in a knowledge graph).

There needs to be some definition of what we want to achieve with interpretability for us to understand what standards we need to keep.

yes! absolutely.

> I think interpretable is a overloaded term.

Author here, this is basically the tl;dr of the paper I kept referencing throughout the post. My take, I hope I was clear, is that understanding the inner workings isn't very helpful, except for ML engineers trying to debug a model.

I think I'd break the terms down something like

- debuggable: The traditional definition of interpretability

- trustable: What I talk about here

I like this point of view, and I'll even go a step further...

If we're going to use LLMs as the basis for anything resembling general intelligence, it won't be through one-shot invocation of the model. It'll be through some kind of chain/tree/graph of thought where the model invokes itself recursively.

In this scenario, we have an exact transcript of the model's thought process, exactly as it occurred, written for us in english. The model can't even have a private "thought", everything needs to be in the visible context.

You can't get more interpretable than that.

A great article - just a technical nitpick for the author: "in-context learning" is not what you do with RAG. "In context learning" is a really confusing name for reasoning by analogy. In RAG you provide source information in the prompt - in ICL you provide examples of how the task should be accomplished. :

"""

In-context learning in language models, also known as few-shot learning or few-shot prompting, is a technique where the model is presented with prompts and responses as a context prior to performing a task. For example, to train a language model to generate imaginative and witty jokes.

We can leverage in-context learning by exposing the model to a dataset of joke prompts and corresponding punchlines:

    Prompt 1: “Why don’t scientists trust atoms?” Response: “Because they make up everything!
    Prompt 2: “What do you call a bear with no teeth?” Response: “A gummy bear!”
    Prompt 3: “Why did the scarecrow win an award?” Response: “Because he was outstanding in his field!”

By training in different types of jokes, the model develops an understanding of how humor works and becomes capable of creating its own clever and amusing punchlines.

"""

from https://www.techopedia.com/from-language-models-to-problem-s...

ah! that's true, I do wrongly conflate those a lot.

If you consider your schizophrenic uncle to be "interpretable", sure.

An alarming trend I see with LLM skeptics is this general direction, where whole categories of real humans are about to be classified as sub-human because an AI has now surpassed their abilities. Every time an LLM skeptic uses the "it's not actually intelligent" argument, it comes dangerously close to dehumanizing actual humans.

This is just you end-running the intelligence debate by assuming LLMs outputs are comparable to humans in anyway beyond the facial

Racist crazy conspiracy-theorist uncle was the example given in the article :)

must read for anyone interested in llms

Reply on news.ycombinator.com