Hacker News Reader

Comment by LASR | Hacker News Reader

I lead an applied AI research team where I work - which is a mid-sized public enterprise products company. I've been saying this in my professional circles quite often.

We talk about scaling laws, superintelligence, AGI etc. But there is another threshold - the ability for humans to leverage super-intelligence. It's just incredibly hard to innovate on products that fully leverage superintelligence.

At some point, AI needs to connect with the real world to deliver economically valuable output. The ratelimiting step is there. Not smarter models.

In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it.

Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.

Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"

We're seeing much less of "it's making mistakes" these days.

If we have open-source models that match up to GPT-4 on AWS / Azure etc, not much point to go with players like OpenAI / Anthropic who may have even smarter models. We can't even use the dumber models fully.

View on HN

> I lead an applied AI research team where I work

Your paycheck depends on people believing the hype. Therefore, anything you say about "superintelligence" (LOL) is pretty suspect.

> Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.

So, what, you're going to build a model to instruct the model? And how do we instruct that model?

This is such a transparent scam, I'm embarrassed on behalf of our species.

This is absolutely on point, I don't understand why your post is being down-voted.

One of the rules of HN is to assume good intent. People do and say the right thing often even when it's antithetical to their source of income. If there is a substantive way to otherwise then say it, don't immediately write-off people's opinions, especially people in their fields because of a possibility of bias.

OK, but the post this was in response to was bloviating about all kinds of sci-fi stuff like "super-intelligence" and the like. It was the opposite of "antithetical to their source of income", instead it was playing into some techno-futurist faith cult.

There is a strong argument that super-intelligence is already in the rear-view mirror. My computer is better than me at almost everything at this point; creativity, communication, scientific knowledge, numerical processing, etc. There is a tiny sliver of things that I've spent a life working on where I can consistently outperform a CPU, but it is not at all clear how that could be defensible given the strides AI has made over the last few decades. The typical AI seems more capable than the typical human to me. If that isn't super-intelligence then whatever super-intelligence is can't be far away.

>My computer is better than me at almost everything at this point; creativity, communication, scientific knowledge

By that logic, libraries, art museums and calculators are "super intelligent"

Are you in the habit of talking to your calculator?

> The typical AI seems more capable than the typical human to me

Your microwave is more capable than a typical human.

If of course your definition of capabilities is narrowly defined to computations and ignores the huge array of things humans can do that computers are no where close to.

> the huge array of things humans can do that computers are no where close to.

For example?

anything involving an interface with the physical world.

For example, running a lemonade stand.

You'd need thousands, if not millions of dollars to build a robot machine with a computerized brain capable of doing what a 6 year old child can do - produce lemonade from simple ingredients (lemon, sugar, water) and sell it to consumers.

Same with basically all cooking/food-service and hospitality tasks, an physical therapy type tasks (massage, chiropractor, etc)...

heck, even driving on public roads still doesn't seem to be perfect, despite 10+ years on investment and research by leading tech companies, although there is also a regulatory hurdle here.

You seem to have shifted the conversation's goalposts there - those are things that computers can do, it just costs a lot.

And, more to the point, they aren't indicative of intelligence. Computers have cleared the intelligence requirements to run a lemonade stand by a large margin - and the other tasks too for that matter.

> those are things that computers can do, it just costs a lot

One could travel between continents in minutes on an ICBM with a reentry vehicle bolted to the front but we don't because it's too expensive. It's a perfectly reasonable constraint to demand that a technology be cost effective. Otherwise it has no practical value.

You forget energy efficiency friend. We are FAR more efficient...

> given the strides AI has made over the last few decades

This is where I lost the plot. The techno-futurists always seem to try to co-opt Moore's law or similar scaling laws and claim it'll somehow take care of whatever scifi bugaboo du jour they're peddling, without acknowledging that Moore's law is specifically about transistor density and has nothing to do with "strides" in "AI".

> whatever super-intelligence is can't be far away.

How do you figure? Or is it just an article of your faith?

It's been the same old, tired argument for seven decades--"These darned computers can count real real fast so therefore any day now they'll be able to think for themselves!" But so far nobody's shown it to be true, and all the people claiming it's right around the corner have been wrong. What makes this time different?

> How do you figure?

https://epochai.org/blog/training-compute-of-frontier-ai-mod...

https://ourworldindata.org/grapher/test-scores-ai-capabiliti...

We're still seeing exponential upswing in compute and we appear to already be probing around human capacity in the models. Past experience suggests that once AIs are within spitting distance of human ability they will exceed what a human mind can do in short order.

These are not subtle trends.

I'm not sure what the amount of computer time spent on training the models has to do with anything, the article states it "is the best predictor of broad AI capabilities we have" without attempting to defend the claim. The "humies" (to use a dated term) benchmarks are interesting but clearly not super indicative of real world performance--one merely has to interact with one of these LLMs to find their (often severe) limitations, and it's not clear at all that more computer time spent on training will actually make them better.

EDIT: re: the computer time metric, by the same token shouldn't block chains have changed the world by now if computer time is the predictor of success? It makes sense for the industry proponents of LLMs to focus on this metric, because ultimately that's what they sell. Microsoft, NVidia, Google, Amazon, etc all benefit astronomically from computationally intensive fads, be it chatbot parlor tricks or NFTs. And the industry at large does as well--a rising tide lifts all boats. It's not at all obvious any of this is worth something directly, though.

> I'm not sure what the amount of computer time spent on training the models has to do with anything

Fair enough. What do you think is driving the uptick of AI performance and why don't you think it will be correlated with the amount of compute invested?

The limitations business looks like a red herring. Being flawed and limited doesn't even disqualify an AI from being super-intelligent (whatever that might mean). Humans are remarkably flawed and limited, it takes a lot of setup to get them to a point where they can behave intelligently.

> EDIT: re: the computer time metric, by the same token shouldn't block chains have changed the world by now if computer time is the predictor of success?

That seems like it would be a defensible claim if you wanted to make it. One of the trends I keep an eye on is that log(price) is similar to the trend in log(hash rate) for Bitcoin. I don't think it is relevant though because Bitcoin isn't an AI system.

1. It's not actually refuting the point being made which is "it's seems hard to take advantage of LLMs' capabilities", which to me seems like a good point that stands on it's own, regardless of who is saying it.

2. The original post is, to my eyes, not implying anything about training a model to help use other models, so the second part seems irrelevant.

I disagree. The comment is striking directly at the claim that people aren't taking "advantage of LLMs' capabilities". This is their capability and no amount of "clear communication" is going to change that

Snarky tone aside, there are different audiences. For example, I primarily work with web dev and some DevOps and I can tell you that the state of both can be pretty dire. Maybe not as much in my particular case, as in general.

Some examples to illustrate the point: supply chain risks and an ever increasing amount of dependencies (look at your average React project, though this applies to most stacks), overly abstracted frameworks (how many CPU cycles Spring Boot and others burn and how many hoops you have to jump through to get thigns done), patterns that mess up the DBs ability to optimize queries sometimes (EAV, OTLT, trying to create polymorphic foreign keys), inefficient data fetching (sometimes ORMs, sometimes N+1), bad security practices (committed secrets, anyone? bad usage of OAuth2 or OIDC?), overly complex tooling, especially the likes of Kubernetes when you have a DevOps team of one part time dev, overly complex application architectures where you have more more services than developers (not even teams). That's before you even get into the utter mess of long term projects that have been touched by dozens of developers over the years and the whole sector sometimes feeling like wild west, as opposed to "real engineering".

That's why articles like this ring true: http://www.stilldrinking.org/programming-sucks

However, the difference here is that I wouldn't overwhelm anyone who might give me money with rants about this stuff and would navigate around those issues and risks as best I can, to ship something useful at the end of the day. Same with having constructive discussions about any of those aspects in a circle of technical individuals, on how to make things better.

Calling the whole concept a "scam" doesn't do anyone any good, when I already derive value from the LLMs, as do many others. Look at https://www.cursor.com/ for example and consider where we might be in 10-20 years. Not AGI, but maybe good auto-complete, codegen and reasoning about entire codebases, even if they're hundreds of thousands of lines long. Tooling that would make anyone using it more productive than those who don't. Unless the funding dries up and the status quo is restored.

I think at its core it's not that there isn't value or future value, but currently there is an assertion, maybe some blind faith, that it's inevitable that a future version will deliver a free lunch for society.

I think the testimonies often repeated by coders that use these code completion tools is that "it saved me X amount of time on this one problem I had, therefore it's great value". The issue is that these all fall into a research of n=1 test subjects. It's only useful information for the subject. It appears we don't realize in these moments that when we use those examples, even to ourselves, we are users reviewing a product, as opposed to validating if our workflow is not just different but objectively better.

The truth lies in the aggregate data of the quality and crucially the speed by which fixes and requirements are being implemented at scale across code bases.

Admittedly, a lot of code is being generated, so I don't think I can say everyone hates it, but until someone can do some real research on this, all we have are product reviews.

> I think at its core it's not that there isn't value or future value, but currently there is an assertion, maybe some blind faith, that it's inevitable that a future version will deliver a free lunch for society.

To me it seems very much like we're somewhere near the peak of the hype cycle: https://en.wikipedia.org/wiki/Gartner_hype_cycle

Except in the case of "AI" we get new releases that seem somewhat impressive and therefore extend the duration for which the inflated expectations can survive. For what it's worth, stuff like this is impressive https://news.ycombinator.com/item?id=41693087 (I fed my homepage/blog into it and the results were good, both when it came to the generated content and the quality of speech)

> The truth lies in the aggregate data of the quality and crucially the speed by which fixes and requirements are being implemented at scale across code bases.

Honestly? I think we'll never get that, the same way I cannot convincingly answer "How long will implementing functionality X in application Y with the tech stack Z for developer W take?"

We can't even estimate tasks properly and don't have metrics for specific parts of the work (how much creating a front end takes, how much for a back end API, how much for the schema and DB migrations, how much for connecting everything, adding validations, adding audit, fixing bugs etc.) because in practice nobody splits them up in change management systems like Jira so far, nor are any time tracking solutions sophisticated enough to figure those out and also track how much of the total time is just procrastination or attending to other matters (uncomfortable questions would get asked them, way too metrics would be optimized for).

So the best we can hope for is some vague "It helps me with boilerplate and repeatable code which is most of my enterprise CRUD system by X% and as a result something that would take me Y weeks now takes me Z weeks, based on these specific cases." Get enough of those empirical data points and it starts to look like something useful.

I think lots of borderline scams and/or bad products based on overblown products will get funded but in a decade we'll probably have mostly those sticking around that have actual utility.

The top comment of your HN link is exactly the issue at hand

> don't know what I would use a podcast like this for, but the fact that something like this can be created without human intervention in just a few minutes is jaw dropping

AI has recently gotten good at doing stuff that seems like it should be useful, but the limitations aren’t obvious. Self driving cars, LLM’s, Stable Diffusion etc are awesome tech demos as long as you pick the best output.

The issue is the real world cares a lot more about the worst outcomes. Driving better than 99% of people 24/7 for 6 months and then really fucking up is indistinguishable from being a bad driver. Code generation happens to fit really well because of how people test and debug code not because it’s useful unsupervised.

Currently balancing supervision effort vs time saved depends a great deal on the specific domain and very little about how well the AI has been trained, that’s what is going to kill this hype cycle. Investing an extra 100 Billion training the next generation of LLM isn’t going to move the needles that matter.

> Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself

Ehm yes. That's because it actually doesn't work as well as the hype suggests, not because it's too "high bandwidth".

I'd contribute that we, the engineering class, are as a whole terrible communicators that confuse and cannot explain their own work. LLMs require clear communications, while the majority of LLM users attempt to use them with a large host of implied context that anyone would have a hard time following, not to mention a non-human software construct. The key is clear communications, which is a phrase that many in STEM don't have the education to know what that phrase really means, technically, realistically.

No, this is absolutely not the reason. The reason is that many people benefit financially as long as the hype train keeps going choo choo. So they lie to our faces and sleep like babies at night.

There is that too, I agree. The hype train is many people's entire careers. But AI does work, is indeed capable of many of the claims, I'm saying the naysayers are using it wrong. But you know who will use it very well? The hype train.

It works for some annoyances in life, though. For example, you can get it to write a complaint to an administration. It's good enough unless you'd rather be witty and write it yourself.

often writing the witty complaint yourself can be not only more effective, but more emotionally satisfying.

Moreover, this is exactly the frustration I've experienced when working with outsourced developers.

Which tells me the problem may be fundamental, not a technical one. It's not just a matter of needing "more intelligence". I don't question the intelligence or skill of the people on the outsourced team I was working with. The problem was simple communication. They didn't really know or understand our business and its goals well enough to anticipate all sorts of little things, and the lack of constant social interaction of the type you typically get when everybody's a direct coworker meant we couldn't build that mind-meld over time, either. So we had to pick up the slack with massive over-specification.

if you can write a specification with no ambiguities you can just execute it, because it's code

Not just with outsourced developers. Computer science graduates hired in house to assist in writing specialised engineering software can often be a dead weight in the team for quite some time. This is not just because they and the engineers don't speak the same language but also because the CS graduates know nothing about physics and engineering so they cannot properly evaluate requirements documents.

Left to their own devices they often implemented requirements that were full of errors or even completely unnecessary because they did not understand the domain well enough to ask pointed questions.

The "it's making mistakes" phase might be based on the testing strategy.

Remember the old bit about the media-- the stories are always 100% infalliable except strangely in YOUR personal field of expertise.

I suspect it's something similar with AI products.

People test them with toy problems -- "Hey ChatGPT, what's the square root of 36", and then with something close to their core knowledge.

It might learn to solve a lot of the toy problems, but plenty of us are still seeing a lot of hallucinations in the "core knowledge" questions. But we see people then taking that product-- that they know isn't good in at least one vertical-- and trying to apply it to other contexts, where they may be less qualified to validate if the answer is right.

I think a crucial aspect is to only apply chatbots' answers to a domain where you can rapidly validate their correctness (or alternatively, to take their answers with a huge pinch of salt, or simply as creative search space exploration).

For me, the number of times where it's led me down a hallucinated, impossible, or thoroughly invalid rabbit hole have been relatively minimal when compared against the number of times when it has significantly helped. I really do think the key is in how you use them, for what types of problems/domains, and having an approach that maximizes your ability to catch issues early.

> Remember the old bit about the media-- the stories are always 100% infalliable except strangely in YOUR personal field of expertise.

Gell-Mann amnesia:

https://en.wikipedia.org/wiki/Michael_Crichton#GellMannAmnes...

* We're seeing much less of "it's making mistakes" these days.*

Perhaps less than before, but still making very fundamental errors. Anything involving number I'm automatically suspicious. Pretty frequently I'd get different answers for the same question (to a human).

e.g. ChatGPT will give an effective tax rate of n for some income amount. Then when asked to break down the calculation will come up with an effective tax rate of m instead. When asked how much tax is owed on that income will come up with a different number such that the effective rate is not n or m.

Until this is addressed to a sufficient degree, it seems difficult to apply to anything that involves numbers and can't be quickly verified by a human.

> Perhaps less than before, but still making very fundamental errors.

Yes.

Suppose someone developed a way to get a reliable confidence metric out of an LLM. Given that, much more useful systems can be built.

Only high-confidence outputs can be used to initiate action. For low-confidence outputs, chain of reasoning tactics can be tried. Ask for a simpler question. Ask the LLM to divide the question into sub-questions. Ask the LLM what information it needs to answer the question, and try to get that info from a search engine. Most of the strategies humans and organizations use when they don't know something will work for LLMs. The goal is to get an all high confidence chain of reasoning.

If only they knew when they didn't know something.

There's research on this.[4] No really good results yet, but some progress. Biggest unsolved problem in computing today.

[4] https://hungleai.substack.com/p/uncertainty-confidence-and-h...

I remember watching IBM's Watson soundly beat Ken Jennings on Jeapordy. One of the things that sticks out most to me about the memory is that Watson had a confidence rating score for each answer it gave (and there were a few questions for which it had very low confidence). I didn't realize it at the time, but that was actually pretty impressive given the overconfidence issues LLMs have nowadays.

Of course, citing sources would always trump any kind of confidence rating in my mind; sources provide provenance, while confidence rating can be fudged just like a bogus answer can.

Providing sources gets interesting to because (a) LLMs have been known to hallucinate nonexistent sources and (b) there isn't a perfect way of knowing whether the source itself was generated by an LLM and contains hallucinations.

Yes. Numbers / math is pretty much instant hallucination.

But. Try this approach instead: have it generate python code, with print statements before every bit of math it performs. It will write pretty good code, which you then execute to generate the actual answer.

Simpler example: paste in a paragraph of text, ask it to count the number of words. The answer will be incorrect most of the time.

Instead, ask it to out each word in the text in a numbered list and then output the word count. It will be correct almost always.

My anecdotal learning from this:

LLMs are pretty human-like in their mental abilities. I wouldn't be able to simply look at some text and give you an accurate word count. I would point my finger / cursor to every word and count up.

The solutions above are basically giving LLMs some additional techniques or tools, very similar to how a human may use a calculator, or count words.

In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values.

The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs.

Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet.

Takes a couple of minutes, but pretty bulletproof and also generalizable to other complex math calculations.

>I wouldn't be able to simply look at some text and give you an accurate word count.

I actually think you would, lots of time I've been surprised by how accurate people can eye-ball stuffs.

> Yes. Numbers / math is pretty much instant hallucination.

Programming is codified applied mathematics and involves numbers in all but the most trivial programs.

> LLMs are pretty human-like in their mental abilities.

LLM's are algorithms. Algorithms do not have "mental abilities", people do. Anthropomorphizing algorithms only serves to impair objective analysis of when they are, or are not, applicable.

> In the products we've built, there is an AI feature that generates aggregations of spreadsheet data. We have a dual unittest & aggregator loop to generate correct values.

> The first step is to generate some unittests. And in order to generate correct numerical data for unittests, we ask it to write some code with math expressions first. We interpret the expressions, and paste it back into the unittest generator - which then writes the unittests with the correct inputs / outputs.

> Then the aggregation generator then generates code until the generated unittests pass completely. Then we have the code for the aggregator function that we can run against the spreadsheet.

How is this not a classic definition of overfitting[0]?

Or is the generated code intentionally specific to, and only applicable for, a single spreadsheet?

0 - https://en.wikipedia.org/wiki/Overfitting

> Anthropomorphizing algorithms only serves to impair objective analysis of when they are, or are not, applicable.

Actually, in this case, comparing how we as humans think to how LLM's work is in fact useful. It's hard for us to eyeball a word and say how many consonants are in it, we need to count. I wouldn't ask a human to eyeball a tax return and tell me what the totals are reliably without giving them the tools to add things up. LLM's are the same way.

It's true that anthropomorphizing in general can be a trap, but when working with LLM's it can be a useful guide in pointing the way towards workable solutions.

> Actually, in this case, comparing how we as humans think to how LLM's work is in fact useful.

Agreed. Contemplating the difference between what people and LLM's are is very useful IMHO. Understanding this is key to making informed decisions as to when LLM's can provide real value.

The assertion originally proffered, however, is quite different than your nuanced perspective:

> > LLMs are pretty human-like in their mental abilities.

It is this to which I object.

I think it would be more accurate to say that LLMs are pretty human-like in their limitations.

> Programming is codified applied mathematics and involves numbers in all but the most trivial programs.

I don't understand that. Can you expand on it?

When does the AI or the programmer need to do math calculations?

Everything a computer does is math...

Yes, but the programmer or the AI don't need to do the calculations in order to program.

>I wouldn't be able to simply look at some text and give you an accurate word count

Actually this ability is at very core of human brain. Yet we humans traded most of it for the ability to speak better. Nevertheless, brain can still read very fast as well as count objects/words really fast, but as you've never really trained, this part of the brain is optimised a lot to do other stuff. Try scanning the texts diagonally, and try to come up with a random word count and what this text is meaning. On the first tries you will be making a lot of errors, but eventually, just 100 hours of training (much less when you are kid) and you will be able to scan the texts in seconds and extract the meaning and the word count very accurately. This is a real technique people use for reading fast.

The thing is – the best and most useful ability of human brain is to adapt, from the very moment it adapted to the harsh reality of being expelled from trees and living on the ground – that's how language was created in the first place, as a result of this adaptation. LLM's can't adapt today, nor doesn't need to adapt, so their mental abilities will never come close to the brain which was adapting for millions of years. Something which is not adapting will become obsolete and come to it's end, this is a fundamental law.

i tell chatgpt i pay 5 dollars for every excellent response, and i make it keep track of how much i spend per chat session (of course in addition to my normal course of work)

it does 2 things. 1 tells me how deep i am in the conversation and 2 when the computation falls apart, i can assume other things in its response will be trash as well. and sometimes number 3 how well the software is working. ranges from 20$ to 35$ on average but a couple days they go deep to 45$ (4,7,9 responses in chat session)

today i learned i can knock the computation loose in a session somewhere around the 4th reply or $20 by injecting a random number in my course work and it was latching onto the number instead of computing $

LLMs don't do math in that sense. They build a string of tokens out of a billion pre-weighted ones that gets a favorable probably distribution when taking your prompt into account. Change your prompt, get a different printout. There is no semantic understanding (in the sense of does what is printed make sense) and it therefore cannot plausibility check its response. A LLM will just print gibberish if it gets the best probability distribution of tokens. I'm sure that's something that will be addressed over time, but we are not there yet.

I'm not keen on marketing words like "superintelligence" but boiling it down that's what in my mind the OP said. These systems are limited in ways that we do not yet fully appreciate. They are not silver bullets for all or maybe even many problems. We need to figure out where they can be deployed for greater benefit.

> Perhaps less than before, but still making very fundamental errors. Anything involving number I'm automatically suspicious

Totally. They are large language models, not math models.

I think the problem is that 'some people' overhype them as universal tools to solve any problem, and to answer any question. But really, LLMs excel in generating pretty regular text.

Problem is also they DO work in SOME cases. e.g. ask ChatGPT to breakdown the calculation to determine tax owed on $xxx of income, and the result is correct in most cases. So there is this perception of intelligence but also fails spectacularly in very simple cases.

In the same way that you can drive a tractor to the shops and sometimes it's a fine mode of transport.

We're seeing much less of "it's making mistakes" these days.

Is this because it's actually making less mistakes, or is it just because most people have used it enough now to know not to bother with anything complex?

They have trained us and we have fine tuned our questions.

> It's just incredibly hard to innovate on products that fully leverage superintelligence.

Once we have actual super intelligence there is no need for humans to innovate anymore. It is by definition better than us anyway.

I guess you could still have artisanal innovation

Today I carved a front panel for my cyberdeck project out of a composite wood board. I hand-drafted everything, and planned out the wiring (though I won't be onto the soldering phase for a while now). It felt good. I don't think having a 3d printer + AI designing my cyberdeck would feel the same.

Yeah I think the whole “humans won’t need to do <insert creative-adjacent-and-skilled-labour-here>” argument misses is the fundamental human aspect.

Computers might do some things “better” than humans, but we’re still going to do things because we _want_ to. Often the fun is in the doing.

LLM’s can vomit out code faster than me, I still enjoy writing software. They’ll vomit out a novel or a song, but I still like reading/listening to stuff a human has taken time and effort to create something, because they can.

Fifteen was a massive success partially because Taylor was only 19 when she recorded it and so it sounded authentic. It's a human singing about her experiences. Also, the lyrics both on macro and micro levels is exceptionally well done and genuinely new.

In other words, in another area: you can take all the art made in the world before Guernica and throw it in any probabilistic network but for sure you won't get Guernica out of that.

Take a look at Nick Bostrom's new book

> AI Creation and the Cosmic Host

> Human civilization is most likely not alone in the cosmos but is instead encompassed within a cosmic host...

Nah, I'm good. I don't need some guy philosophizing about AI being our pathway to join a galactic super-civilization.

When AI is actually doing something useful for humanity, call me back.

Get back to me when you actually have superintelligence ;)... I'll wait.

> In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it.

It's a token prediction machine. We've already generated most of the ideas for it, and hardly any of them work because see below

> Getting AI to do work involves getting AI to understand what needs to be done from highly bandwidth constrained humans using mouse / keyboard / voice to communicate.

No. Gettin AI to work you need to make an AI, and not a token prediction machine which, however wonderful:

- does not understand what it is it's generating, and approaches generating code the same way it approaches generating haikus

- hallucinates and generates invalid data

> Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"

Indeed. Instead of asking why, you're wildly fantasizing about running out of ideas and pretending you can make this work through other means of communication.

> hallucinates and generates invalid data

The model is doing the exact same thing when it generates "correct" output as it does when it generates "incorrect" output.

"Hallucination" is a misleading term, cooked up by people who either don't understand what's going on or who want to make it sound like the fundamental problems (models aren't intelligent, can't reason, and attach zero meaning to their input or output) can be solved with enough duct tape.

I like the analogy of an Ouija board: "This magical conduit to supernatural forces sometimes gives out nonsense, but soon we'll fix it so that it always channels the correct ghosts and spirits."

It was "cooked up" by people like Joshua Maynez, Shashi Narayan, et al, who I'm going to guess understand what is going on, and adopted by the rest of the field.

https://aclanthology.org/2020.acl-main.173/

I like the term because it's concise and describes the output :)

But true, it's just the output.

> Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"

I find this funny because for what I use ChatGPT for - asking programming questions that would otherwise go to Google/StackOverflow - I have a much better time writing queries for ChatGPT than Google, and getting useful results back.

Google will so often return StackOverflow results that are for subtly very different questions, or ill have to squint hard to figure out how to apply that answer to my problem. When using ChatGPT, i rarely have to think about how other people asked the question.

We have ideas on how to leverage it. But we keep them to ourselves for our products and our companies. AI by itself isn’t a breakthrough product the same way that the iPhone or the web was. It’s a utility for others to enhance their products or their operations. Which is the main reason why so many people believe we’re in an AI bubble. We just don’t see the killer feature that justifies all that spending.

> In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it.

This is just another way of saying this technology, like Blockchain before it, is a solution in search of a problem.

Or, more drastically, another way of saying it’s useless.

Your use of the word superintelligence is jarring to me. That's not yet a thing and not yet visible on the horizon. That aside, the point I like that you seem to be making is along the lines of: we overestimate the short term impact of new tech, but underestimate the long term impact. There is a lot to be done and a lot of refinement to come.

Have we surpassed Clippy "It looks like you're trying to write a letter" yet?

Yes, this is in my mind where people will find the fabled moat they search for too.

SOTA models are impressive, as is the idea of building AGIs that do everything for us, but in the meantime there are a lot of practical applications of the open source and smaller models that are being missed out on in my opinion.

I also think business is going to struggle to adapt and existing business is at a disadvantage for deploying AI tools, after all, who wants to replace themselves and lose their salary? Its a personal incentive not to leverage AI at the corporate level.

>Anyone using a chatbot already has felt the frustration of "it doesn't get what I want". And also "I have to explain so much that I might as well just do it myself"

the problem is really, can it learn "I need to turn this over to a human because it is such an edge case that there will not be an automated solution."

I.e. long story short - are you saying that this tech is a solution looking for a problem?

Makes sense.

"In my mind, already with GPT-4, we're not generating ideas fast enough on how best to leverage it."

This is the main bottle neck, in my kind. A lot of people are missing from the conversation because they don't understand AI fully. I keep getting glimpses of ideas and possibilities and chatting through a browser ain't one of them. On e we have more young people trained on this and comfortable with the tech and understanding it, and existing professionals have light bulbs go off in their heads as they try to integrate local LLMs, then real changes are going to hit hard and fast. This is just a lot to digest right now and the tech is truly exponential which makes it difficult to ideate right now. We are still enveloping the productivity boost from chatting.

I tried explaining how this stuff works to product owners and architects and that we can integrate local LLMs into existing products. Everyone shook their head and agreed. When I posted a demo in chat a few weeks later you would have thought the CEO called them on their personal phone and told them to get on this shit. My boss spent the next two weeks day and night working up a demo and presentation for his bosses. It went from zero to 100kph instantly.

Just the fact that I can have something proficient in language trivially accessible to me is really useful. I'm working on something that uses LLMs (language translation), but besides that I think it's brilliant that I can just ask an LLM to summarise my prompt in a way that gets the point across in far fewer tokens. When I forget a word, I can give it a vague description and it'll find it. I'm terrible at writing emails, and I can just ask it to point out all the little formalisms I need to add to make it "proper".

I can benchmark the quality of one LLM's translation by asking another to critique it. It's not infallible, but the ability to chat with a multilingual agent is brilliant.

It's a new tool in the toolbox, one that we haven't had in our seventy years of working on computers, and we have seventy years of catchup to do working out where we can apply them.

It's also just such a radical departure from what computers are "meant" to be good at. They're bad at mathematics, forgetful, imprecise, and yet they're incredible at poetry and soft tasks.

Oh - and they are genuinely useful for studying, too. My A Level Physics contained a lot of multiple choice questions, which were specifically designed to catch people out on incorrect intuitions and had no mark scheme beyond which answer was correct. I could just give gpt-4o a photo of the practice paper and it'd tell me not just the correct answer (which I already knew), but why it was correct, and precisely where my mental model was incorrect.

Sure, I could've asked my teacher, and sometimes I did. But she's busy with twenty other students. If everyone asked for help with every little problem she'd be unable to do anything else. But LLMs have infinite patience, and no guilt for asking stupid questions!

> I can benchmark the quality of one LLM's translation by asking another to critique it

Do you speak two or more languages? Anyone that does is wary of automated translations, especially across estranged cultures.

> It's a new tool in the toolbox, one that we haven't had in our seventy years of working on computers,

It's data analysis at scale, and reliant on scrapping what humans produced. A word processor does not need TB of eBooks to do it's job.

> They're bad at mathematics, forgetful, imprecise, and yet they're incredible at poetry and soft tasks.

Because there's no wrong or right about poetry. Would you be comfortable having LLMs managing your bank account?

> If everyone asked for help with every little problem she'd be unable to do anything else.

That would be hand-holding, not learning.

> Do you speak two or more languages? Anyone that does is wary of automated translations, especially across estranged cultures.

I'm aware that it's imperfect. It's still pretty cool that they're multilingual as an emergent property - and rather than merely translating, you can discuss aspects of another language. Of course it hallucinates - that's the big problem with LLMs - but that doesn't make it useless. Besides, while automated translators aren't perfect, they're the only option in a lot of situations.

> It's data analysis at scale, and reliant on scrapping what humans produced. A word processor does not need TB of eBooks to do it's job.

And I'm not replacing word processors with LLMs, nor did I claim that they were trained in a vacuum.

> Because there's no wrong or right about poetry. Would you be comfortable having LLMs managing your bank account?

No, which is why I don't intend to... they're probabilistic and immensely fallible. I wasn't claiming they were gods. My point was that they're far outside of what we usually use computers for (e.g. managing your bank account), and that opens a lot of possibilities.

> That would be hand-holding, not learning.

Correct. That's why we don't do it... and in normal circumstances, it'd probably be better to deeply consider the problem in order to work out why you were wrong. But a week before the exam it's excellent. I got a healthy A*, so it doesn't seem to have hurt.

What exactly are you arguing against? I'm not convinced they're a route to AGI either, and I'm not about to replace my graphics driver with an LLM, nor code written by one - but you seem to have a vendetta against them that's lead to you sidestepping every single point I made with a snide remark against claims you seem to have imagined me making.

In some ways being human is also data analysis at scale and being reliant on what others have produced...

What was the demo? And what is the advantage of integrating local LLMs vs third party?

I will be glad to see the day when LLMs will be able to play Minecraft, so I won't have to. Then I can just relax and watch someone else do everything for me without lifting a single finger.

Won't that be fun.

Watching people play Minecraft is the main hobby of children these days

Well, I guess you've caught me then. Naturally, adults would be more preoccupied with how they can use AI to improve their lives and get ahead, instead of playing Minecraft.

why do you believe that humanity is the habitat for ai and not the other way around?

This comment goes a long way to explaining Jonestown to me.

We have a new god (superintelligence) but only the special can see it because it's too intelligent for people to interact with it.

It's so advanced all the common ways humans communicate "using mouse / keyboard / voice" don't work

The reason we've never seen it help us is we need more ideas (prayers) first.

NPC brains are really hard to understand I will say that and they use a lot of electricity.

Reply on news.ycombinator.com