Hacker News Reader

Comment by letmevoteplease | Hacker News Reader

There is one: "The agreement between GPT-4 and humans reaches 85%, which is even higher than the agreement among humans (81%). This means GPT-4’s judgments closely align with the majority of humans. We also show that GPT-4’s judgments may help humans make better judgments. During our data collection, when a human’s choice deviated from GPT-4, we presented GPT-4’s judgments to humans and ask if they are reasonable. Despite different views, humans deemed GPT-4’s judgments reasonable in 75% of cases and are even willing to change their choices in 34% of cases."[1]

[1] https://arxiv.org/abs/2306.05685

View on HN

That's helpful!

I've done a lot of work in audio synthesis, which is notoriously difficult measure. The gold-standard is human ratings of audio quality, but it is tough to design good tests (easy to fatigue raters) and the iteration time waiting for results is quite long.

Instead, there's now some projects which use neural networks trained on human ratings to predict audio quality, such as ViSQoL: https://github.com/google/visqol

This opens up fast iteration - scores going up generally corresponds to higher quality - followed by human testing at major milestones (eg, releasing a paper/model). VISQOL has a harder time comparing 'unrelated' models, IMO - ends up being not so great for comparison of different techniques, but excellent for measuring incremental improvement or catching regressions.

But, in the end, yes - you can use NN's to measure the quality of other NN's, so long as you're careful about it and make use of human raters from time to time as well.

The problem of test data getting into the training data seems to be an especially pernicious issue with LLM's, which isn't really arising in the audio synthesis space.

I’ve started doing this with ASR hypotheses from colloquial spontaneous speech. It tends to have similar issues. Lots of shady human ground truth especially where addresses, alphanumeric sequences, repairs and repetitions and other essentially non read speech are concerned. The very large Whisper models are consistent in their transcription style and highly reliable as long as you pick strongly represented languages. And ChatGPT can do a very good job at comparing the linguistic coherence of hypotheses from multiple recognizers. Together these models can annotate, analyze and ingest far more data more consistently than human annotators at this point (at least in the best covered languages). We haven’t quite realized this as a community yet though, because the standard datasets we use for evaluation contain all these human inconsistencies. Wild times.

just curious, are there any open models doing the opposite of audio synthesis? As in able to generate the stems for a song?

https://github.com/facebookresearch/demucs

Ha, I was working on neutral voice compression, which consists of an encoder which creates what the kids are calling tokens these days, and a decoder which synthesizes speech from the tokens.

AudioLM puts a language model on top of the compression tokens, and thus can generate speech or other audio. There's piles of recent papers pushing that approach into music generation. Mulan is a name that comes to mind.

Or maybe your interested in audio separation to get at the isolated instruments? There's lots of great work on that, as well. Like MixIT, which is an unsupervised audio separation system.

It's funny how ChatGPT really does give you the most balanced, middle of the road answers. It feels like a distillation of all human knowledge and sentiments. I use it constantly to get advice on plans, architectures, thoughts, etc.. to get an idea of pretty much what the average person would think. It often points out things I've overlooked which I'll improve my design with and go back and forth with ChatGPT until we're both in agreement.

I even read a classic book the other day and had a great discussion with ChatGPT about moral relativism, the different schools of thought and how it fit into philosophy as a whole. For students this technology is incredible, I wish I had it for all my classes.

Even sometimes comments I'll make on here or Reddit I'll pass through ChatGPT first to see if I made any mistakes in my logic.

> to get an idea of pretty much what the average person would think

There is no such thing as an average person.

https://www.thestar.com/news/insight/when-u-s-air-force-disc...

Funnily enough I think you both might be right here, there isn't such a thing as an average person, but ChatGPT may be the synthesis of the average opinion.

what is an average opinion? it is the sum of opinions which disagree with the result

Maybe "balanced" rather than "average" is a better way of putting it?

What’s the balance between, say, opinions that trans people should be exterminated vs not? What’s the balance between Ukraine sovereignty vs Russian occupation? Etc

My general experience (not that topic) is that for anything even slightly approaching the sides of the Silicon Valley Overton Window, ChatGPT creates micro-essay saying "On the one hand, $foo, on the other hand, $bar. It's important to remember that $topic is controversial, and that many people disagree."

It would involve serious consideration of the opinions you don't agree with, rather than just qualifying them in the most hyperbolic and dismissive way possible. Hopefully ai is better capable of this sort of reasoning than people are.

It gives you the average opinion of someone on the Internet, particularly places like Reddit, which is FAR from the “average opinion” of most people. This has been a problem for a while, people keep assuming the Internet represents some kind of moral or ethical consensus on so many issues when it’s not even close.

On the other hand, since AIs are taught using user content from Reddit, once other cultures start using these AIs, they will begin to conform more towards those norms (or will actively go against it and The Great Sort will continue forward).

The other way to phrase this is high-dimensional spheres are "spikey".

Would put slight caution around asking it anything more than "what should I read next about this?"

For whatever reason*, it is particularly bad at discussing philosophy I find. When I was grading philosophy 101, I would have probably given it a passing grade against the overall curve, but that's about it. Philosophy is a discipline of careful, sometimes jargoney, and always very couched assertions that can be easily misunderstood and appropriated. This is probably its greatest weakness, and in many ways this weakness is the progenitor of philosophy itself in the Western world, with Plato at the start (i.e. with the figure of the sophist, the paradox of a false wisdom).

- Maybe one reason: there is a huge amount of, lets say, "armchair philosophy" on the internet, compared to other disciplines. Many blogposts and tiny manifestos of people really excited by some out of context quote from Spinoza or whatever. And you start to really feel this part of the dataset when you ask about philosophy. Many strange takes and misunderstandings.

I asked it where moral relativism fit in with philosophy and it came back with this

    Philosophy
        Ethics/Moral Philosophy
            Meta-Ethics: The study of moral thought, language, and properties
                Moral Realism: Belief that there are objective moral facts
                Moral Anti-Realism: Denial of the existence of objective moral facts
                    Moral Relativism: The belief that moral judgments are true or false only relative to some particular standpoint

I thought that was pretty good, what do you think?

Haha I think it's fine. I think its kinda cheeky answering you so literally, giving it an actual place to fit into :).

I don't doubt it can do, like, Wikipedia type classification ok, but that's not like really getting to the substance of anything! And, either way, its not like there is one decided-upon hierarchy of concepts like this people consciously work within. This is a fine picture to some, but others might contest, perhaps, that Meta-Ethics is the "study of moral thought, language, and properties." What is "moral language" anyway? Why is it meta relative to Moral Philosophy writ-large? Or perhaps one might argue that we need to think of meta-ethics as a sibling rather than child. The whole discipline is a mess of different thoughts and possible rebuttals and grand intellectual overturnings that will not be captured here. Maybe just try pasting that back into the prompt and asking "what's wrong with this picture?".

But like I said, its fine in that its fairly comparable to Wikipedia for utility, (with IMO a worse interface, but I get why people like it more).

Yea I don’t think it was trained on any hierarchy in particular, ChatGPT literally came up with this on the fly. There’s even more to it I didn’t include to keep the post small. Just really impressive to have a conversation with it that goes all sorts of places. I don’t know any philosophy professors so this is the next best thing.

The reason you feel that way might be that you are familiar enough with philosophy.

After all, LLMs and ChatGPT in particular are are indistinguishable from productised Gell-Mann Amnesia.

Edit: rewrote to be more neutral, sorry

> what the average person would think

Keep in mind that due to the nature of the data and the RLHF training, it's more like a weighted average, something like

    0.5*(average American view) + 0.4*(average WEIRD view) + 0.1*(average human view).

(where WEIRD = Western, Educated, Industrialized, Rich, and Democratic, standard terminology in psychological research.)

This may or may not matter depending on the questions you're asking, just something to keep in mind.

80% agreement is high, but the margins between models at the top are so low that even that remaining 20% could be enough to alter the final rankings, depending on which direction it errs.

Not just that, but are the 80/20 randomly distributed? Probably not. These comparisons might have more in common with the 20%.

But 80% agreement is the same as between humans

True, but given that you're using one of the models to judge the others, it's likely that the cases of disagreement will tend to favor GPT-4. You would never use one of the competitors as a judge among humans.

If only there were some way to produce some kind of "interval" of scores where you were confident that the actual score sat, and then had some way of comparing these intervals between the different models...

Is perfect agreement possible? And what is the definition of agreement? Humans don't agree about much.. are we saying agreement means it matches the truth after intensive investigation by humans?

Ugh, when I was doing my PhD work we were studying creativity in an experiment, and we needed an assessment for how creative different solutions were, and trying to get inter-rater reliability on this quite simple thing was just agonizing.

I wound up abandoning the experiment because getting enough reliability would have required screwing down the standards so tightly that it would have ruined the underlying point of the thing.

Kind of hard to consistently evaluate creativity when someone might pull a James T. Kirk (https://en.wikipedia.org/wiki/Kobayashi_Maru), which is only creative the first time and just a cheat thereafter.

Just one of many problems :)

It's surprising to me that you describe creativity as a simple thing. How were you defining and measuring it?

I don't (and didn't) think creativity was simple, but the task was super simple (alternate uses task). The fact that people couldn't agree on how creative the answers were to such a simple task was the insight, although maybe it was only an insight because I was naive.

If it's trained by humans is it safe to assume that we'll get it so something crazy like 99% agreeable?

Reply on news.ycombinator.com