OpenAI Threatening to Ban Users for Asking Strawberry About Its Reasoning

that can’t be true, we were assured that their vague worldview about “doing things that benefit humanity” based on “just being a good person” would be strong enough to overpower the urge to dominate and profit immensely.

In a plot twist, MOLOCH was never necessary because a corporation of people who relinquish ethical responsibility to do everything and anything for personal gain was already in charge under the name... CORPORATRON.

Grandparent was referencing the classic Man Behind the Curtain twist (AI behind the curtain?). Human leadership might very well be listed, but in that view they're all already knowingly or not controlled by the AI.

Ah yes, noted independent director Sam Altman.

Might be a blessing for introverts. Just turn on some autopilot chat giving generic responses while I can zone out in my thoughts.

The inevitability sounds like people will stop bothering participating in social networks, and just participate with their own virtual world produced by their local AI. I mean… in the end there will be no difference in output

But it is quite profitable.

To be fair a huge reason openAI was created was because google had LLM's but wasnt letting anyone use them. Remember the google researcher who claimed his ai was sentient? Obviously wasnt true but that product was in no way open to the public.

I stand corrected. He's a con man.

I feel that drop outs are presented as if they are beyond education, they’re too smart to waste their time in school. While that could be true in theory, I resist taking that for granted. You can always look at it the other way around as well: unable to finish things or looses interest quickly. It’s all a meaningless interpretatin after all but all I can say is for sure that Mr Altman does not have a degree.

It's not name calling, it's a description. He is actively selling tech he doesn't fully understand, that is fundamentally not capable of doing what he is selling it to do. LLM's have a place, they have for decades at this point, but they are not intelligent, not in the way he goes out of his way to evoke with what he says, and certainly not in the way his audiences believe. I don't know enough to have a proper opinion on whether true AI is possible; I have to assume it is. But nothing OpenAI has shown is that, or has the potential to be that, nor is it worth anything near 150 billion dollars. It's an overvaluation among overvaluated companies making up an overvaluated industry and when it goes, not if, when, it will have ramifications throughout our industry.

I'm sure it won't though for Mr. Altman. He's done a fantastic job failing-up so far and I have no reason to assume this will be any different.

What else would you call it?

I mean I've said it before and I'll say it again: you do not need AI images/video for faking political spin shit. That was done LONG before chatGPT/Midjourney/Sora existed because the people you're targeting want to believe that Obama eats babies or whatever the fuck, they don't need a 4K photo-realistic rendering of it to buy in, you just tell them he's doing it and they're on board.

People still believe GamerGate was about ethics in journalism when we have screenshots of the posts where the dickheads responsible for it made fun of the people they would trick with that lie before they told it and they still believe it. Motivated reasoning is a hell of a drug.

Job security and financial safety

> Any other person trying to pull that off would be in jail already.

Not ANY other person. Just people who are not rich and well-connected. See also: Donald Trump.

> We never would've gotten GPT-3 and GPT-4 if this didn't happen.

"We never would've gotten [thing that exists today] if [thing that happened] didn't happen", is practically a tautology. As you saw from the willingness of Microsoft to throw compute as well as to hire ex-OpenAI folks, as you can see from the many "spinoffs" others have started (such as Anthropic), whether or not we would've gotten GPT-3 and GPT-4 is immaterial to this discussion. What people here are asking for is open AI, which we might, all things considered, have actually gotten from a bona fide non profit.

Maybe all companies are doomed this way, but it was the first step on a slippery slope. Not in terms of the slippery slope logical fallacy, that's only apply if someone argued they'd end up force-hiding output before GPT-3 if they went capped profit

>We never would've gotten GPT-3 and GPT-4 if this didn't happen.

That doesn't justify fraud, for instance.

Unfortunately, people are becoming increasingly illiterate with regards to what is legal and what is not.

> We never would've gotten GPT-3 and GPT-4 if this didn't happen

Well, of course. But we'd get similarly powerful models elsewhere. Maybe a few weeks or months later. Maybe even a few weeks or months earlier, if, say, OpenAI sucked up a lot of talent and used it wastefully, which I don't find implausible at all.

If they want to be for-profit because that's how they get the investment to build GPT-4, fine, do it from the start. That doesn't justify the switch.

Wrong.

There's rules to follow to prevent what is called "private benefit", which OpenAI most likely broke with things like their (laughable) "100X-limited ROI" share offering.

>It is a pretty common model [...]

It's not, hence why most people are misinformed about it.

This is misleading at best. There are rules you must follow to do this legally and OAI's structure violates some of them and is under scrutiny from the IRS so their new plan is for the non-profit to completely sell off the subsidiary and then die or go into "maintenance mode" with the new fully commercial subsidiary carrying the ball (and the team) forward to riches.

I considered things like this as an original Mozilla person back in the day. Mozilla could have sold the Firefox organization or the whole corporation for billions when it had 30% of the web, but that would have been a huge violation o of trust so it was never even on the table.

That so many here are fans of screwing the world over for a buck makes this kind of comment completely unsurprising.

So does OpenAI, the nonprofit, own a for-profit corp that does everything?

Open collaborative work is great. But a complement, not a replacement, for high resource work.

Nobody needed OpenAI for normal collaborative open work to thrive.

Their original idea was a relatively uncommon idea: attempt to combine (1) pushing the absolute front forward, with high resource work that matched or exceeded work from deep pocketed corporations, while (2) sharing and collaborating. They did both for a while, resulting in as significant impact as anyone could have hoped form.

But those two dimensions are hard to maintain together. It requires a continuous stream of vast donations.

The latter is very difficult, perhaps completely unrealistic.

Burger King didn’t start with royalty.

Adobe didn’t start as clay producer.

Apple didn’t start as fruit selling company.

Open in their name was not randomly chosen to sound fun, it’s a reference to well known concept. It’s “young democrats” becoming far right but keeping their original name.

where are you getting that information that it's not the meaning of the word that OpenAI talked about originally? Did they ever promise that the model weights, or exact architecture would be open? I would like to see that promise. What exactly in your opnion did they promise that they have violated?

> open - in some sense.

The phrase that leaps to mind is "Open Beta."

"In some sense", any word can mean anything you want. "Open" carries with an accepted meaning in technology that in no way relates to what you're describing. You may as well call McDonald's "open".

What they're doing is almost literally the opposite of EEE. If OpenAI actually had open models, then Facebook could take those models, add their own non-open things to the models, and use this new stuff as a business advantage over OpenAI. Instead, they're independently developing their own models and releasing them as open source to lower the value of proprietary models.

https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

Like Y2K, there's an argument that diligence from the tech crowd prevented this.

> But seriously, if you paid attention over the last decade, there was so much shit about big tech that people said were going to lead to tyranny/big brother oversight

To be fair, the big tech controls the behavior of the people now. With social media algorithms and by pressuring everyone to live in social media. Existence of the many companies depends on the ads on (NAMEIT) platform. Usually the people with most power don't have to say it aloud.

The usual "GPL is anti-freedom" argument is that it restricts what someone is allowed to do with the source code, meaning it is less free than MIT or BSD style licenses.

I don't agree with that, but that is what the person is saying.

What's absurd, in my opinion, is lumping GPL advocacy in with two other tropes which are intended to restrict the sharing of information and knowledge, where GPL promotes it.

I see that sentiment largely coming from developers who, I think, misunderstand the freedom that the GPL is protecting.

The GPL focuses on the user's freedom to modify any software they are using to better suit their own needs, and it does a great job of it.

The people saying that it is less free than bsd/mit/apache are looking at it from a developer's perspective. The GPL does deliberately limit a developer's freedom to include GPL code in a proprietary product, because that would restrict the user's freedom to modify the code.

To save someone a search: POSIWID = The purpose of a system is what it does

https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...

Even that doesn't agree with reality. Some tyrants prefer to say they're doing things for your freedom, your money, or your rights. (In fact, you know what, it's silly to expect tyrants to stay consistent.)

My point is that spending $50 billion training an AI is a reckless thing to do, so OpenAI and its competitors need to be stopped.

It's reckless because the AI might turn out to be better at planning and better at reality than our most capable institutions (e.g., the FBI and the military) so there would be no way to stop it from doing whatever it wants, and there is no plan that anyone has published or proposed that might prevent the AI from wanting something incompatible with continued human survival.

And most things are incompatible with continued human survival if enough optimization pressure is applied to bringing the thing about, so it is unlikely we will get lucky and the AI ends up wanting something compatible with continued human survival.

If someone somewhere comes up with a viable plan before the end, then the labs could probably be persuaded to follow the viable plan (so we would be saved) because the leaders of the labs understand at some level that what they are doing is very dangerous, and they don't want to be killed, but it is unlikely that anyone anywhere is going to come up with a viable plan in time, so the labs are going to stick with the clearly inadequate plans they have now.

The reason I think it is unlikely that anyone anywhere is going to come up with a viable plan is that MIRI (then called the Singularity Institute for Artificial Intelligence) starting the search for a viable plan in about 2002 and the 2 people at MIRI (Yudkowsky and Soares) with the most experience in this search both say it is unlikely that anyone is going to come up with a plan before the end unless there is a multi-decade moratorium on AI research and on very large training runs.

You sound like a lazy parent. Reap what you sow.

If YouTube actually rots your brain then it would be stupid to watch it yourself. So the kid would be making a solid point.

However the actual issue for me is that YouTube tries to keep you endlessly captivated with content regardless of quality or value. So it can ends up being a time sink, which at the end you feel like nothing of value was consumed.

So I tell my kids that, not that it rots their brain. And when there’s content we can both enjoy, we can watch it together. Or if they want to watch something specific on YouTube, they can watch that thing and then be done.

Look up knowledge graph. Its an old technique superseded by LLm

o1 gets this correct, through pure reasoning without a program. OP was likely using GPT-4(o|o-mini)

I think it'd be just too expensive to incorporate code-writing in CoT. Maybe once they implement having a cluster of different model sizes in one answer it'll work out.

>This alone is an indictment of their "reasoning" capability.

I'm not sure why it says anything about their reasoning capability. Some people are blind and can't see anything. Some people are short-sighted and can't see objects which are too far away. Some people have dyslexia. Does it say anything about their reasoning capability?

LLMs "perceive" the world through tokens just like blind people perceive the world through touch or sound. Blind people can't discuss color just like LLMs can't count letters. I'm not saying LLM's can actually reason, but I think a different way to perceive the world says nothing about your reasoning capability.

Did humans acquire reasoning capabilities only after the invention of the alphabet? A language isn't even required to have an alphabet, see Chinese. The question "how many letters in word X" doesn't make any sense in Chinese. There are character-level LLMs which can see every individual letter, but they're apparently less efficient to train.

Would an LLM using character tokens perform better (ignoring performance)?

This is the correct take.

Much has been written about how tokenization hurts tasks that the LLM providers literally market their model on (Anthropic Hiaku, Sonnet): https://aclanthology.org/2022.cai-1.2/

yes, you are missing that the tokens aren't words, they are 2-3 letter groups, or any number of arbitrary sizes depending on the model

The fact that any of those tasks at all work so well despite tokenization is quite remarkable indeed.

You should ask why it is that any of those tasks work, rather than ask why counting letter doesn't work.

Also, LLMs screw up many of those tasks more than you'd expect. I don't trust LLMs with any kind of numeracy what-so-ever.

The problem is not the addition, is that the LLM has no way to know how many r's a token might have, because the LLM receives each token as an atomic entity.

For example, according to https://platform.openai.com/tokenizer, "strawberry" would be tokenized by the GPT-4o tokenizer as "st" "raw" "berry" (tokens don't have to make sense because they are based on byte-pair encoding, which boils down to n-gram frequency statistics, i.e. it doesn't use morphology, syllables, semantics or anything like that).

Those tokens are then converted to integer IDs using a dictionary, say maybe "st" is token ID 4663, "raw" is 2168 and "berry" is 487 (made up numbers).

Then when you give the model the word "strawberry", it is tokenized and the input the LLM receives is [4463, 2168, 487]. Nothing else. That's the kind of input it always gets (also during training). So the model has no way to know how those IDs map to characters.

As some other comments in the thread are saying, it's actually somewhat impressive that LLMs can get character counts right at least sometimes, but this is probably just because they get the answer from the training set. If the training set contains a website where some human wrote "the word strawberry has 3 r's", the model could use that to get the question right. Just like if you ask it what is the capital of France, it will know the answer because many websites say that it's Paris. Maybe, just maybe, if the model has both "the word straw has 1 r" and "the word berry has 2 r's" and the training set, it might be able to add them up and give the right answer for "strawberry" because it notices that it's being asked about [4463, 2168, 487] and it knows about [4463, 2168] and [487]. I'm not sure, but it's at least plausible that a good LLM could do that. But there is no way it can count characters in tokens, it just doesn't see them.

It doesn't see "straw" or "berry". It sees a vector which happens to represent the word strawberry and is translated from and to English on the way in and out. It never sees the letters, 'strawberry' is represented by a number, or group of numbers. Try to count the Rs in "21009873628" - you can't.

This is outside the bounds of the point OP was making, but FWIW an image is ultimately just a grid made up of pixels. You need only to loop your pixel classifier over the 2d array and tabulate.

My experience has been Claude is better at all coding/technology questions and exploratory learning. It's also night/day better at less common tech/languages than others (try asking ChatGPT questions about Svelte, for example).

Claude is also vastly better with creative writing (adcopy) and better at avoiding sounding like a LLM wrote it. It's also vastly better at regular writing (helping you draft emails, etc).

We were using OpenAI's Teams for a while. Tried Claude out for a few days - switched the entire company over and haven't looked back.

OpenAI gets all the hype - but there are better products on the market today.