Hacker News Reader

Comment by rabbits_2002 | Hacker News Reader

rabbits_2002

It is crazy to me that we have actually reached a point where you just tell a computer to do something, and it can

It's even crazier to me that we've just... Accepted it, and are in the process of taking it for granted. This type of technology was a moonshot 2 years ago, and many experts didn't expect it in the lifetimes of ANYONE here - and who knew the answer was increasing transformers and iterating attention?

And golly, there are a LOT of nay-sayers of the industry. I've even heard some folks on podcasts and forums saying this will be as short-lived and as meaningless as NFTs. NFTs couldn't re-write my entire Python codebase into Go, NFTs weren't ever close to passing the bar or MCAT. This stuff is crazy!

I think this is the point where the field has just entered pseudoscientific nonsense.

If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag

Interesting! What do you think it's a red flag for? If it's easier to use the prompt why not do that at the level of abstraction that makes sense?

It means we don’t understand the mapping between the model and the prompt-level semantics. It also means we can’t cleanly separate (1) controlling the conversational behavior of the model from (2) having a conversation with the model. It means we have a black box that we can experiment on, but we can’t plan or predict how to control and customize it, nor can we reliably predict and control its behavior, because we can’t accurately reason about it.

None of your conclusions are true. We can plan and predict just fine with it. It’s not fully deterministic, but still more predictable than people. And there are a lot of things we don’t understand but are fairly predictable, like the use of pain medications. They aren’t 100% predictable, but good enough to be able to administer as part of a well thought out regimen.

And given the fact that we can test it in a somewhat closed system gives us much more ability to predict its behavior than many things “in real life”.

Sure, it's a sign that we don't "understand this stuff properly", but you can say the same about human brains. Is it a red flag that we use language to communicate with each other instead of manipulating nerve impulses directly?

We have no choice but to have and use our brains, not so with LLMs. We don’t have to start building core technologies off of fundamentally flawed models.

Yeah, if we wanted to flap wings to fly, we would have never got the airplane.

The ornithopter is real, mind you.

> fundamentally flawed

Citation needed

Great point. Btw: The problem is corporate irresponsibility:

When self-driving cars were first coming out a professor of mine said "They only have to be as a good as humans." It took a while but now i can say why that's insufficient: human errors are corrected by discipline and justice. Corporations dissipate responsibility by design. When self-driving cars kill, no one goes to jail. Corporate fines are notoriously ineffective, just a cost of doing business.

And even without the legal power, most people do try to drive well enough to bit injure each other which is a different calculus from prematurely taking products to market for financial gain.

Very succinctly put - this captures my view but I couldn’t have put it in these words; thanks!

Human errors are not corrected by discipline and justice. Drunk driving is still a huge problem.

The top 3 causes of death by vehicle accident in USA are [0]:

- DUI

- speeding

- distraction

In other words all human errors. Machines don’t drink, shouldn’t speed if programmed correctly, and are never distracted fiddling with their radio controls or looking down at their phones. So if they are at least as good as a human driver in general (obeying traffic laws, not hitting obstructions, etc.), they will be safer than a human driver in these areas that really matter.

What do you care more about—that there is somebody specific to blame for an accident or that there are less human deaths?

0: https://www.idrivesafely.com/defensive-driving/trending/most... and many other sources you can find

Under corporate control safety spirals down to increase profit. See: opiods, climate change, pesticides, antibiotic resistance, deforestation, and privacy. 50 years from now self-driving cars will be cheaper and more dangerous. Human driving misbehavior will still be disincentivized through the justice system, but corporations will avoid individual responsibility for dangerous programming.

We could likely put 80% of cars on rails just as easily

They only have to be as good as humans because that's what society deems an acceptable risk.

I do think the point about how companies are treated vs humans is a good one. Tbh though, I'm not sure it matters much in the instance of driver-less cars. There isn't mass outrage when driver less cars kill people because that (to us) is an acceptable risk. I feel whatever fines/punishments employed against companies would only marginally reduce deaths, if that. I honestly think laws against drunk driving only marginally reduce drunk driving.

I'm not saying we shouldn't punish drunk driving... just that anything short of an instant death penalty for driving drunk probably wouldn't dissuade many people.

In my country, drunk driving is punished by losing license and banning you from using another one for half year for first time and of life for second. And it's very effective, as those cases are rarity now

> It took a while but now i can say why that's insufficient: human errors are corrected by discipline and justice.

If they did, we'd be living in utopia already.

But also, by the same token, generative AI errors are similarly "corrected" by fine-tuning and RLHF. In both cases, you don't actually fix it - you just make it less likely to happen again.

Such a strange take. We have no choice but to use our brains??? It is also an incredibly capable system! At some point if the capabilities are so drastically different is it confusing that you would choose a much more capable system even with all its flaws?

Because you are demanding and may rise up with pitchforks if the corporate class asks too much.

At the same time, humans are also unreliable as hell , push us much more than 8 hours a day of actual thinking work and our answers start to get crappy.

Are you suggesting we just ignore the immense capabilities of these models because we don't fully understand them?

I hope you never need any medicine!

> Is it a red flag that we use language to communicate with each other instead of manipulating nerve impulses directly?

From an almighty creator's, yes. Direct telepathic communication is much more efficient compared to spoken language. Just look at the Protoss and where they went, leaving us behind :-(

> but you can say the same about human brains

It should be an HN rule that in order to type out variations of this sentence you have to also prove you have a degree in neuroscience.

I think you should have to show your ML creds to claim LLM research is "pseudoscience", but here we are.

Well it's not a science and you're going to have to a hard time convincing me that Richard Feynman is wrong.

You don't have to have a degree in neuroscience to be aware of the fact that we aren't even close to understanding how human brain works when it comes to high-level process such as sentience.

First off we understand a lot about the human brain, but that doesn't matter because no one is arguing that we understand the totality of the human brain, instead what is being argued is: The human brain is as simple as an LLM and thus LLMs are sentient, can reason, and can know facts.

Yeah, I wonder how much of the same could be achieved by just adding a few keywords.

Like in Ultima Online, where people could either do "Dear Sir, may I buy your wares?" which would be equivalent to "buy wares".

vendor buy the bank guards

It's not pseudosience if the prompts are engineered according to the scientific method: formulate a hypothesis, experiment, reincorporate the results into your knowledge.

But it's a very fuzzy and soft science, almost on par with social sciences: your experimental results are not bounded by hard, unchanging physical reality; rather, you poke at a unknown and unknowable dynamic and self-reflexive system that in the most pathological cases behaves in an adversarial manner, trying to derail you or appropriating your own conclusions and changing its behavior to invalidate them. See, for example, economics as a science.

> But it's a very fuzzy and soft science, almost on par with social sciences

There's no need for name-calling.

I think what you're asking for--this total scientific control and understanding of how the model will behave was never going to happen for any models close to our own intelligence.

I think thats a bit pessimistic. I am sure it would be possible to bake these prompts into the model but if this is equally effective and much less effort to do it this way, why is it an issue?

The crazy part is that it became banal so fast that people act like it's no big deal and will absolutely insist that the computer doesn't "actually" understand anything because it hasn't reached some perfectly interpretable platonic ideal of understanding yet, and anyone amazed that you can talk with the pile of sand is just being naive.

Technically that’s always been the case. It’s just that now you can tell the computer what to do in a “natural language” like English instead of a “programming language”.

We have reached a point where the computer no longer does exactly that we tell it to do :P I always though of computers being stupid, but very fast. With AI they are slow, and make human errors :P

With the right GPU they can make mistakes almost at the speed of thought.

I've been writing software for 35+ years, and the computer has frequently done something other than what I thought I told it to.

Turns out that I'm not as good at telling computers what to do as I'd like to think.

And now we intentionally tell the computer to ignore what it is told if it’s isn’t deemed proper by the mega corps.

And if it won’t, you might be able to weasel your way into it with a sob story about your grandma.

This isn't necessarily a uniform improvement, it is however, a great novelty at present. Time will tell.

I mean, yes it's cool I can write an essay to get Dall-E to generate almost the exact image I want (and I still can't using natural language), is it truly an improvement ? Yes I can churn out content faster, but I can't make the computer draw exactly what I want with words alone.

A picture is worth a thousand words, or maybe two thousand?

It does something, not definitely the thing you asked

Reply on news.ycombinator.com