Meta AI claims to have a child in a NYC public school | Hacker News Reader

Meta AI claims to have a child in a NYC public school

twitter.com

66 pointschedar41 comments

The problem here isn't that an LLM hallucinates. The problem is that nobody asked for an AI response, and Meta pushed content to a forum that makes such claims, which could easily mislead or at least confuse people not sophisticated enough to be on the lookout for hallucinations.

Meta should be (and is) in the business of policing third-party spam on their forums that does exactly this. We can infer what must've happened - the model must've been fine-tuned on forum comments, and this would be the likely format for a response to that question. This sort of thing should've been caught by a wrapper/guard model, and will probably make a good case to add to such a model's instructions/training.

(btw: is it "an LLM" or "a LLM"? I guess I should ask an LLM which it prefers to be called)

In regards to your last question, it’s “an LLM”.

The distinction between using “a” vs “an” is one based on the immediately proceeding syllable sound rather than the letter. If it’s proceeded by a vowel, then use “an”, and if it’s proceeded by a consonant, use “a”.

Because “LLM” is pronounced “el el em”, the first syllable sound is “eh”—a vowel.

The same letter may need different “a”/“an” article based on how the word is pronounced. For example “an LLM” vs “a layer”.

See this for someone smarter than me explain it: https://www.merriam-webster.com/grammar/is-it-a-or-an

As someone with a photographic memory as my main (because it's most the "performant" one I have) learning tool, I strongly feel that somebody should make a really long list of most commonly used words which should use "an", and conversely, a list of words which should use "a" of a similar length (both as two columns of text). I'm not really eager to "run" the check (mentioned under the link) in my head every time I need to choose an article, and the biggest problem, and the reason behind some of the mistakes in choosing the right article I sometimes do, is that I don't really see some the less common words which should use "an" (despite starting with a letter which suggest otherwise) this often, or actually, often enough.

And when I'm reading a text for any other purpose than memorizing article choices (so, 99,999% of cases), they don't get enough of my attention to get remembered - it's the meaning of other words which get it, and big part of which will get remembered, not the articles used before them.

Being able to look on such a list every few days for say, a month, would definitely help to remember most of these cases.

For an abbreviation, it's the first letter's pronunciation. You could easily make a list of those 26.

For an acronym, it's the word's pronunciation, and is (almost?) always the same as the standard rule: Vowels vs consonants.

"Almost" can make a big difference when writing stuff where formal style is expected. For instant messaging, yeah, probably one doesn't have to care.

When writing, you put articles before a letter, but they're based on what phoneme they precede. Therefore, when purely classifying letters, there's much more combinations than 26. In hundreds or lower thousands, possibly. Or more.

That's the difference, and for me it's frankly to memorize a big look-up table of most commonly used words (and which articles should precede them), because this doesn't require any effort to me, than to run an "algorithm" translating to phonemes every time I write something. In a quick reading (not reading out aloud, or even mentally mimicking reading something out aloud), wrong article being used won't necessarily get easily picked up. It's the sheer laziness, I guess.

Such a table can't exist. Pronunciation is varies, so your choice of articles adds character to your text in much the same way your accent does for spoken words.

Take "herb," for example. In some dialects, the "h" is vocalized, while in others, it's silent. Both "an herb" and "a herb" are valid. Your choice in your writing conveys identity. An author who opts for "a herb" helps paint a vague picture of the individual behind the words, perhaps someone from England.

You could make your own personal table, but it would be for you and only you.

Also, although there is a concrete rule, it's not something we're thinking about as we talk--using the wrong article just feels wrong. Most of us aren't consciously "running an algorithm," as you put it; the correct article just comes out.

Most people will find that they develop the same skill with writing over time. The subset of people who have trouble developing that skill and learn best by memorizing a table of words is going to be quite small. I would never write "a LLM" in the same way that I would never say "an history" out loud.

That's a really good comment! Thanks. By the way, pronunciation of articles also varies. With some variants "a" would get much more universal (but also making it harder to notice when it's used wrong), while with others it would be almost impossible to use "a" where "an" should be used without exposing yourself to a major tongue-twister.

And yeah, I'm aware that the subset of people mentioned is quite small. On this subject, it's that memorization requires close to no effort for me, and is close to instantaneous and long-lasting (as long as I run through it several times and the things learned aren't ending up being completely unused), while developing the intuitive feel of the right article, as you rightly put it, takes time (however, can also be close to effortless to some)... and lots of writing.

I'm pretty sure "an" is used for vowels and "a" is used for consonants, but I might be wrong

There's more than one problem, and one of them sure is that a "helpful AI" hallucinates.

LLMs are Internet simulators. That is a simulation of a good response you would get on the Internet.

Is anyone still surprised by this? If so, let me repeat: LLMs are Internet simulators. They will give you simulations of good replies you might get on the Internet.

I’d be skeptical of a claim like this.

But how to prove its not true?

how about everyone stop calling it AI when it very clearly is little more "intelligent" than "intelligent device for starting your heater when it gets cold" (aka thermostat) (and yes, this has really been marketed)

It's been co-opted into the phrase "AI" in the same vein that "cloud" was used in "Cloud Computing". It's a marketing term to make the technology more appealing. That's why half the people started using the term "AGI" and even with that we see the tech companies leveraging the hype the marketing teams are bringing in while selling the illusion that building a computer smarter than a human brain (something we still barely understand) is feasible in our lifetimes or ever.

(sorry I am just venting my annoyance with it too, I don't mean for this to come off as hostile if it does!)

Also the nanotechnology blockchain gig economy for self-driving cats.

At least "E-Commerce" turned out to be real, eventually.

I’ve given up on correcting people. “AI” has been erroneously used to describe LLMs enough that it’s in most peoples’ heads as the correct usage. Language changes over time, and even if we’re “right” about the term AI being incorrectly applied here, we don’t get to make choices about how language is used - it’s its own beast and it obeys its own laws.

The makers of Velcro (brand name) would love for people to use “hook and loop fasteners” when referring to velcro (generic), but once enough people are using a term you might as well try to fight the tide.

That would be called a feedback mechanism?

As someone who has seen "person of interest",... I believe it.

And that child’s name?

dell-32a7iZ

It'd be funnier if that was a valid Dell service tag. ;)

Ok, and?

You’re not worried about the child’s well-being? Monster.

Well, at least the kid can get a meal at school; sure not getting anything at home.

If that's a child of an "AI", I'm not sure does they even need "food" in form of carbohydrates, fats and proteins (or amino acids) that humans need. Electricity is all they need, probably. Or maybe even directly it's light, if they're an optical computing device. Or if you purely regard them as a software thing, computing power is all they need.

Much lesser needs of dwelling space, too.

The kid could not make the cut for private school, so...

"Meta AI claims"

No it doesn't. It can't. Only people (or companies, which require people) can meaningfully "claim" things. LLMs are still not people, despite our persistent attempts to personify them.

This is merely a sexier headline than "Hallucination machine hallucinates." And even that word personifies a bit too much!

What verbiage would you deem acceptable, if "claims" and "hallucinates" are both out?

“Weighted random text generator generates random text that doesn’t make logical sense” maybe. Seems to be both accurate and better describes what’s actually going on. The AI can’t claim or hallucinate anything because that requires it to have core beliefs and senses that can be fooled by incorrect inputs.

It's right in the most famous system's name: "Generative Pre-trained Transformer". LLMs generate text (and other media) which imitates their pre-training corpus. That's all.

At the end of the day, it's turning a mathematical crank. LLMs have no more intentionality than a jack-in-the-box.

You're right in a purist sense, but the alternative headline would be "Meta is already carelessly allowing hallucination machine to hallucinate in troubling places and with troubling amplification by their content placement algorithm"

It's definitely a vivid example of Meta being irresponsible with the tech today and of what we can expect a lot of the internet to be polluted with in the future.

I understand your point but I think it's misplaced here. This kind of criticism made sense with the Bing/Sydney stuff, since "Bing claims to be self-aware" is misleading compared to "Bing inconsistently recites some sci-fi cliches." People like Kevin Roose were actively spreading misconceptions about how LLMs work. It would have been nice if they had taken your comment seriously in 2023.

But this is different. The subtext of the headline is clearly "Facebook's dumb chatbot had a very dumb glitch." I believe laypeople would immediately understand the AI is just plagiarizing a Facebook mom. Policing the language here seems more about pedantry than correcting actual misconceptions.

(You might say that some people would read this headline and jump to a Her fantasy, where Meta's poor AI is desperate for human connection or whatever. But these people are not going to be swayed by technical accuracy. They will just interpret language like yours as euphemism and denial.)

Any more information on this?

I heard so much about the Kevin Roose stuff, is there a breakdown somewhere of what actually happened.

From the way that podcast presented it, Microsoft had the bing bot untethered in a way that it kepted taking in more and more context and was just taking it correctly.

This is against my current much less virgin, but very much simple, understanding of how llms/gpt works.

What actually happened there?

> Policing the language here seems more about pedantry than correcting actual misconceptions.

Feel free to blame me, if it helps. I've got broad shoulders.

However the fact that we're (collectively) losing the mass "mindshare battle" doesn't imply bad faith. Some of us are still fighting the good fight, and I don't see a problem with that.

Personally I think this only means we should fight harder against these dangerous beliefs, not throw in the towel (or worse, friendly fire against fellow educators).

And yes, it's human-side beliefs that are dangerous, not the tech itself. If an LLM "suggests" to kill <group of people> and we know what an LLM really is, then it's harmless. However if a large fraction believe an LLM is some infallible AI oracle or genie (a surprisingly common belief), then this "suggestion" could cause catastrophic harm.

At this point, the reputational risk that one malfunctioning generative AI has on _all_ generative AI is concerning.

The reason this occurs as a narrative is because hallucination is a widespread issue with LLMs, and thus can be fairly generalised across most deployments of them.

Talking to some disconnected sets of people who are much more layman and not computer users, when I speak of some cool new AI app, they look at me with eyes of perversion and somewhat disgust with fear.

The way the information has been coming out and sold of ai to pump up stock prices is to the detriment of public opinion. Curious to see how things go.

It’s not AI it’s an llm. Word frequency autocomplete on steroids.

How do you define intelligence?

Know it when I see it. Goodhart's law shows that any other definition will be gamed.

That's awfully convenient.

For whom is it inconvenient?

Reply on news.ycombinator.com