It's inevitable that we'll reach AGI. It's inevitable that humans will extinct.
Everything you described is not how today's AI works. It's not even a stretch, it's just pure sci-fi.
Why wouldn't it? It's trained on code generated by humans and already generates code that is more readable than the output of many humans me included.
You can fairly easily update the “next token” choice with a syntax check filter. LLMs like ChatGPT provide a selection of “likely” options, not a single perfect choice. Simply filter the top-n recommendations mechanically for validity. This will improve output a lot.
Similarly, backtracking can be used to fix larger semantic errors.
Last but not least, any scenario where a test case is available can be utilised to automatically iterate the LLM over the same problem until it gets it right. For example, feed it compiler error messages until it fixes the remaining errors.
This will guarantee output that compiles, but it may still be the wrong solution.
As the LLMs get smarter they will do better. Also, they can be fine tuned for specific problems automatically because the labels are available! We can easily determine if a piece of code compiles, or if it makes a unit test pass.
We're very much in the "early days" of experimenting with how LLMs can be effectively used. The API restrictions enforced by OpenAI are preventing entire categories of use-cases from being tested.
Expect to see fine-tuned versions of LLaMA run circles around ChatGPT once people start hooking it up like this.
It will have to describe these requirements in a way that a human can understand, and verify. The language will have to be unambiguous and structured. A human will need to be able to read that language, build up a mental model, and understand it is correct, or know the way to make corrections. Who do you think that person will be? Hint: it will be a specialist that knows how to think in a structured, logical way.
edit: Ultimately there are going to be iterative pipelines with traditional programmers in the loop rearranging things and reprompting. Math skills are going to be deemphasized a bit and domain skill value increased a bit. Also, I think there's going to be a rise in static analysis along with the new safe languages, giving us more tools to safely evaluate and clean up output.
"Everything's broken, why am I paying you?"
"Everything works, why am I paying you?"
I believe the world is wiggly, not geometrically perfect, intellectuals struggle with that because square problems are easier to solve. Ideal scenarios are predictable and it’s what we like to think about.
Have you ever had to use a sleep() intentionally just to get something shipped ? That’s a wiggle.
We’re going to try square out the world so we can use ChatGPT to solve wiggly problems. It’s going to be interesting.
Yesterday I tried to use a SaaS product and due to some obscurity my account has issues and the API wouldn’t work, they have a well specified API but it still didn’t work out, I’ve been working with the support team to resolve it, but this is what I call a wiggle, they seem to exist everywhere.
Ask a construction worker about them.
Hah. So true. The more I work on renovating parts of my house the more I see where a workers experience kicked in the finagle something. Very analogous to programming. All the parts that fit together perfectly are already easy today. It’s those bits that aren’t square, but also need to fit where the ‘art’ comes in.
Can AI also do that part? IDK, currently I believe it will simply help us do the art part much like the computer in Star Trek.
If we need a semi-intelligent system to help us with the copy pasta, so be it.
It failed spectacularly
I wonder if it's because the API is quite large, and I had to paste in ~10 messages worth of API docs before I was finished.
It kept repeating segments of the same routes/paths and wasn't able to provide anything cohesive or useful to me.
Was your API pretty small? Or were your docs pretty concise?
GPT-4 can now accept 8k or 32k. The 32k version is 8 times larger than the one you tried.
And these advances have come in a matter of a few months.
Over the next several years we should expect at least one, quite easily two or more orders of magnitude improvements.
I don't believe that this stuff can necessarily get a million times smarter. But 10 times? 100? In a few months the memory increased by a factor of 8.
Pretty quickly we are going to get to the point where we have to question the wisdom of every advanced primate having a platoon of supergeniuses at their disposal.
Probably as soon as the hardware scales out, or we get large scale memristor systems or whatever the next thing is which will be 1000 times more performant and efficient. Without exaggeration. Within about 10 years.
That should be the place for experiments like this.
Lowery latency links back to Earth and first see how it goes.
Also you don’t think there will be resource constraints at some stage ? It’s funny we yelled at people for Bitcoin but when it’s ChstGPT, it’s fine to run probably tens of thousands of GPUs? In the middle of a climate crisis ? Not good.
Also, with BTC it's literally burning it in an unproductive way for "improved security". It's like lighting a forest on fire to keep warm.
All the AI tools combined, last I heard, aren't consuming 0.5% of the world's energy usage. And even if they were, it would be absolutely bonkers to argue we should keep doing that when there were alternatives that accomplished similar goals without the energy usage (proof of stake)
It's really the early days, but there's no way energy consumption won't grow exponentially now there is potential for earning money.
You need a real ability to reason and preserve context beyond inherent context window somehow (we humans do it by keeping notes, writing emails, and filing JIRA tickets). So while this doesn't require full AGI and some form of AI might be able to do it this century, it won't be LLMs.
no, I'm not that deep in hell
But it's definitely not true of the average piece of software. So much of the world around us runs on software and hardware that somebody had to build. From your computer itself, to most software that people use on a day-to-day basis to do their jobs, to the cars we drive, to the control software on the elevators we ride, software is everywhere.
There is a lot of waste in software, to be sure, but I really don't think the average SE works for a company that shouldn't exist.
But upper management can know exactly what LLMs are capable of, because they are products with fixed capabilities. ChatGPT is the same ChatGPT for everybody. This makes firing obsolete workers much safer.
Everyone thinks only in terms of current needs and state of affairs of people when analyzing a future technology. No one thinks about the insatiable human desire for more and the higher expectations for that new normal that always meets the increased productivity available. Anything that automatically solves much of our wants is doomed to be static and limited.