Hacker News Reader

Comment by icyfox | Hacker News Reader

Congrats on the launch!

I have an interesting dissonance with this. On one hand, I understand how huge parameter sets can and do model specific personas well. I've also read some of these cited papers and _know_ intellectually that predicted results can be close to actual survey data. The other part of me is screaming at my laptop that language modeling is about aggregate statistics, revealed preference counts for a lot, and how could a language model actually substitute for market research?

I imagine the biggest hurtle you're going to face are research teams that:

- A. Want to see actual proof behind data

- B. Disbelieve a LLM could generate statistically significant insights about real people that would make individual decisions

- C. Need to justify their own existence / organizational clout with boots on the ground facilitating surveys

A and C might be surmountable, but I'm not sure of a good way of tackling B.

View on HN

Thank you!

Agree A, B, and C are big hurdles.

Re: A - we have started adding transparency (vis-a-vis the 'Investigate Results' and the tSNE plots + similarity scores) but we still have a ways to go

Re: B - agree that the survey responses -> insights pipeline is nonlinear and it's not clear how to make that tighter

Re: C - generally, we try to champion a human-AI interaction loop where people are needed to evaluate the outputs, generate insights, etc.

All great points though and ones we are facing

>The other part of me is screaming at my laptop that language modeling is about aggregate statistics

Yeah..but it's not. This is where people are having so much trouble. The erroneous belief that language model learn some "average" or "aggregate" of the training set. But when you get down to it, that doesn't make any sense at all. What help would some fictional aggregate distribution do with predicting the responses of decidedly non aggregate text ? None at all.

So Language models don't learn an "average" distribution. They learn to predict every state that exists in the corpus in a sort of superposition. The perfect LLM would predict Einstein as well as it would predict the dumbass down the street.

LLMs are biased but not uniformly so.

I think we're using aggregate statistics in two different ways.

Technically speaking - the whole idea with a language model is that you're learning to generalize underlying patterns of text, not just memorize the input text. Otherwise language models would be very good at echoing back training data but fail miserably during validation. If we go back to the training sequence - it's trying to maximize the posterior given the conditional probabilities in the sequence:

P(y1, y2, ... yn ) = P(y1) * P(y2|y1) ... P(yn|y1...yn-1)

That probability is by definition an aggregate; it's the best fit given potentially competing inputs of the training set that all have the same input conditional chains.

Where generative LLMs have a leg-up is because they have such a large parameter space, large context windows, and coherent sampling strategies. This helps them stay internally consistent with their response data. But at the end of the day what they are learning are still patterns. That's why they aren't able to link content back to the exact source of origin; parameters fuse inputs from different places into one hybrid.

Seeding a generative chain with Einstein or someone down the street doesn't change the fact that what's next is some fused learning from a lot of different training set inputs.

The point i'm making is that i believe the "fused space" is not in fact very fused because that would be directly detrimental to reducing loss.

Reply on news.ycombinator.com