Comment by submeta | Hacker News Reader

Comment by submeta | Hacker News Reader

Parent Full thread

To me, ChatGPT is the most exciting thing that I have come across in a long time. - I can't stop finding use-cases for it. Examples

- Want a book summary quickly, without going through dozens of google search results? Try "Summarize Gilgamesh Epos" or "Summarize Carl Sagans Daemon Haunted World". => You can even ask a summary for each chapter of a book, if it has a record of that book.

- Want a quick overview about what DAO in crypto space means? "Explain DAO in the context of crypto"

- Want to get a quick overview of the old testament? "Give me an overview of the parts of the old testament, and describe in detail what the books are about"

- Want to compare Buddhism to Christianity? "Compare buddhism to christianity"

- Want a hypothetical conversation between Socrates and Napoleon, where Socrates tries to convice Napoleon that war is futile? "Create a fictional dialog between Socrates and Napoleon where Socrates tries to convice Napoleon that war is meaningless."

- Want a quick bio of Ataturk, in French?

- Want to have a quick overview about French Revolution, and how it effected the world? What other events were relevant during that time?

Even better: Want it to produce results in the Q&A style so that you can copy/paste it in your Ank app? "Create Questions and Answers about French Revolution so that I can use it in my Anki App. End each Question and Answer by triple X and a newline."

There are endless use-cases where I don't have the time to go ask google, collect the results, review and read them, to answer my question. This already replaces Google for me for these kind of quick Q&A sessions.

This is extremely exciting stuff, and I can't even imagine what we'll have in ten years from now.

While I agree, I have found ChatGPT to be REALLY awful at symbolism because so many people are as well. It gives popular responses rather than most correct responses. As we have seen from politics, popular often times isn't correct.

An example of this is music:

ChatGPT responds to "What is the meaning of The Postal Service's The District Sleeps Alone Tonight?" With: "The meaning of "The District Sleeps Alone Tonight" by The Postal Service is open to interpretation, as the lyrics are somewhat vague and abstract. However, the song appears to be about the feeling of isolation and disconnection that can come from living in a big city. The lyrics reference the city as a "church" that "wraps [the narrator] in its twilight," and speaks of the planes flying overhead and their passengers "waving goodbye/ To all of their yesterdays." This could be seen as a metaphor for the transitory and fleeting nature of city life, with people constantly coming and going and not forming deep connections. The song may also be about the narrator's own sense of loneliness and longing for connection in the midst of the bustle and noise of the city."

There's two issues with this response: 1. The song is about the weird feeling of how places change with context, in this case it is about the context of a breakup and feeling out of place where you used to feel at home.

But there is a bit of a bigger issue.... 2. Those lines ARE NOT IN THE SONG. lol.

What this shows is clean readable responses that are oftentimes extrapolation of a thread or idea to levels not reasonably verifiable. Like how Facebook is/was tuned for giving more and more isolated extreme content, ChatGPT is tuned to follow threads and bits of information to extreme levels and confidence.

I think the "confident but inaccurate" problem is one of the biggest hurdles for most of the powerful usecases of these models. We don't seem too far from being able to automate filler text or stock images, which is certainly useful but only so valuable, but a long way off from more technical usecases like law or medicine. Being right 90% of the time but completely wrong 10%, without any way to distinguish the two, isn't going to get very far in many fields of work.

I had it give me book recommendations on a particular topic. It straight up made up a fake book with a fake author and attributed it to a publisher. It did this with confidence. I even emailed the publisher to see if they had ever carried that book. Never heard of it.

The problem seems to be whether or not the content was generated via language modeling or via direct references. When it crosses into language modeling it is just making stuff up on the fly. This sometimes works for programming, but for specific things that connect to the real world it can be phenomenally wrong.

The thing is, accuracy doesn't matter that much for some applications, and for other applications it matters a lot. This means we should evaluate ChatGPT on the former use cases where it actually has utility, instead of criticizing it on the latter which would be like saying a hammer isn't helping me cook my dinner.

Example of the former: using it as a conversation partner to learn another language. You don't need perfect accuracy for this to be a gamechanging tool.

good taste :P

"Explain X (an acronym) in the context of Y" has become by go-to use case for chatgpt, super good and cuts through a lot of the nonsense you run into when trying to pose the question to google.

Reply on news.ycombinator.com