Hacker News Reader

Don't bother trying their other models available in GCP either. Their embedding models and their un-RLHFd generic GPT analogues are miles behind the competition. It's incredible how bad google have dropped the ball on this.

We had some google people come in at $CORPO_DAYJOB the other day to sell their cloud offerings. The engineer had the gall to say "people ask me, why did google miss the boat on LLMs? I say people, we BUILT the boat". While referencing "attention is all you need". Hilarious coping strategy guys, but because you had some good academics on your payroll in 2017, that doesn't mean you're delivering right now... we'll talk when you actually have something better than 90% of the other LLM offerings out there.

LLMs are in their infancy. ChatGPT is bleeding users by an alarming amount. It's incredible how bad OpenAI have dropped the ball on user retention, product integration and revenue.

I'm not discussing, let alone contesting user retention, revenue or integration. I'm making an observation on my perception of raw model output.

So the raw model output doesn't affect user retention, product integration and revenue? The quality of the raw output is probably why it's failing so fast and hard.

Building the boat and missing the boat are totally compatible. They guided others to a place they could not enter themselves.

It's also compatible with what I know about the business side of things. Google Brain/Research probably spent a lot of time and money in the neural network direction and still didn't have a clear way to productize it, so they cut them off just before they got there. Other companies were able to start where they left off since the research was public and make something with less time - making the investment in that direction look more attractive to higher ups.

Yeah, I would say that "we built the boat" is simply a lie given none of the eight authors of the paper still work at Google.

That’s a strange take. Those researchers were given the resources, the right incentives, the right environment for that kind of research to happen, at Google, in a time when LLMs weren’t all the rage.

“X Built a boat == the builders remain at X until they die” Is a strange definition of building something imo.

I would say it's still correct.

When a company provides an R&D lab, we still attribute the results of that research to the company as well as to the researchers.

After all, if those individuals were not hired that position would still be filled with someone similar, but it's hard to argue that if the lab didn't bring those researchers together, they may have individually gotten the same result.

Honestly, it's not as easy as one may think to build a research lab.

> As a point of comparison, I posed the exact same question to OpenAI's ChatGPT-4. It responded with a spot-on account of the event, complete with rich detail and an impressive level of accuracy.

Which, most likely, implies that chatgpt was trained and retains actual book content rather than random people talking about it on the internet.

Which model is more likely to be in violation of some copyright?

The ChatGPT one, but that isn’t my problem. My concern is whether it gives me good information. The issue of copyright is an unfortunate issue for someone else to litigate.

Question, do you think this has to do with copyrighted information being in the models ? I know you might not care but I wonder if Google is operating of accounts from the internet while OpenAI has actually ingested the books ?

No.

A blog post on the internet is also copyrighted! Google has probably the largest collection of digitized books as well.

They have no reason not to. ML networks have been trained on copyrighted data since before 2012.

> They have no reason not to.

Ethics?

Personally, I have no ethical issue with it.

But you can still read and gather information from it. Your computer still stores a copy of it when you reload the page. Theft? Infringement? I think not.

Is taking the character count of a book copyright infringement? Why is the math behind an LLM different?

> Is taking the character count of a book copyright infringement? Why is the math behind an LLM different?

I hate arguments like this. Intent, scale, and quantity matter.

If I gently toss a small piece of lead at you, I'm not trying to kill you. If I accelerate it using a rifle, I am.

No, taking the character count of a book is not copyright infringement. Neither is quoting the word "the". Or "the worst". But, "It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair." definitely is.

The fact that you have to draw a line and exercise some amount of judgement to determine which actions are prohibited does not mean the line doesn't exist.

Likewise, the social context and emergent consequences of a policy matter too.

If LLMs were a niche technology used by a couple of fringe weirdos, who cares, go nuts. But since LLMs are owned by large tech companies and are being used to generate incredible amounts of revenue, they are enacting a large-scale transfer of wealth to the rich whose value is largely derived from the labor of the creative people who authored the data the model is trained on without any consent on their part.

How you can't see that that is somehow different from taking the word count of a file is beyond me.

Perhaps it would be a reason if they actually thought it was unethical.

I don't know but I think google bard had access to the book's content as well because it mentioned correctly some characters and some places I didn't mention in my question.

That's exactly what LLMs are good at. Cross-referencing information from across the training data.

The response could contain information that you did not specifically refer to but is relevant and/or related. It doesn't even imply that the LLMs had access to the book's content.

Good point. If I ever want a random summary of an event in a fantasy book I've already read, I'll be sure to stick to ChatGPT.

On a completely unrelated point, why are people's tests of LLMs clearly designed to make me think that people are dumber? Are the LLMs suggesting these questions as part of their plot?

Great. And if I want your opinion on how smart it is to be a bit silly with LLMs when I’m not even attempting to test them systematically, I’ll be sure to ask you.

What's it got to do with you?

Anyone here has ideas on what could make a difference between openai and bard on this issue ? Not an expert, so i really have no idea what it is that makes this kind of difference. training ? Architecture ? Rlhf ? Etc

What was the prompt?

In the book the The game of Thrones what happens at Saltpans with Arya and Sandor Clegane ?

Here the link with the answer of ChatGPT: https://chat.openai.com/share/7c46a3bd-efa8-49e9-8018-554e1e... and also a follow up question.

Is this actually correct? https://awoiaf.westeros.org/index.php/Saltpans makes me think ChatGPT is referencing the wrong book.