I yawned so hard my jaw unlocked.
Can't wait to see groundbreaking... checks notes... "advancements in various applications, optimizations, and extensions of the model".
Do these companies only hire yes men?
https://www.microsoft.com/en-us/research/wp-content/uploads/...
And actually there is no need to go as far as “universe” to get to something that can’t be captured by language. Human existence is such an example.
For this reason I don’t think llms are going to be good film makers for instance. Sure an llm will be able to spit the scenario of the next action movie, those already seem to be automatically generated anyway. But making a film that resonates to humans takes a lot that can’t be formulated with language.
This is both extremely powerful and limiting.
An LLM is never going to give you some of the most famous films like "Star Wars" which bounced around before 20th Century Fox finally took a chance on it because they thought Lucas had talent. Is what we want? A society that just uses machines to produce variations of the same thing that already exist all the time? It's hard enough for novel creative projects to succeed.
Yes, state of the art models like midjourney, sd3 are _really_ good. You are bounded only by your imagination.
The idea that generative AI is only derivative was never an empirical claim, its always been a cope.
Because you focus on how they are similar and not how they are different, to me it is extremely obvious they are very different. Students make mistakes and learn and then stop doing them soon after, when I taught students at college I saw that over and over. LLM however still does the same weird mistakes they did 4 years ago, they just hide it a bit better today, the core different in how they act compared to humans is still the same as in GPT-2 to me, because they are still completely unable to learn or understand their mistakes like almost every human can.
Without being able to understand your own mistakes you can never reach human intelligence, and I think that is a core limitation of current LLM architecture.
Edit: Note that many/most jobs doesn't require full human general intelligence. We used to have human calculators etc, same will happen in the future, but we will continue to use humans as long as we don't have generally intelligent computers that can understand their mistakes.
So far as I know, all current AI need far more examples than we do.
But, that's not why LLMs are "unable" to learn: the part which does that is simply not included in when it's deployed for inference.
Maybe others can complete it, maybe it'll be easy to complete it in twenty years, with a little more hindsight. Maybe.
Ok, but that's more on you than on current AI; the models which get distributed (both LLMs and Stable Diffusion based image generators) are already found in re-trained and specialised derivatives created by people who know how to and have a sufficiently powerful graphics card.
The weights are frozen on purpose. You can "thaw" them.
I don't know what you mean by that.
If you mean qualia, then sure. Unsolved and undescribed. But other than that, I think everything has a linguistic form; perhaps inefficient, but it is possible.
Separately, transformers don't have to use what humans recognise as a langue, this means they can use things such as DNA sequences and pictures. They're definitely not the final answer to how to do AI, because they need so many more examples than us, but I don't have confidence that they can't do these things, only that they won't.
But on the philosophical side, if an understanding can’t be communicated, does it exist? We humans only have various movements and vibrations of flesh, sensing those, text, and images to communicate.
There are deep mathematical results about our limits to understand things simply because we communicate through finite series of symbols from finite dictionaries. Basically what we can express and prove is infinite but discrete, but there is much larger infinities than that that will be beyond our grasps forever. Things like theorems that are true but can not be proven to be true, or properties on individuals real numbers that exist but can not be expressed.
And there is no reason to believe the universe doesn't have the same kind of thing: it remains to be shown whether or not you can describe or understand the universe with a finite set of symbols.
It begs the question, if sifting through noise is a meaningful way to look for scientific progress. And of course, what if it's wrong? Both the Library of Babel and AI are fully capable of leading us down untested and nonsense rabbit-holes. The difference between Alice and Wonderland and Jabberwocky is unknown to us; we wouldn't know which books are worth reading and which are not.
On the one hand, you have people excited by this idea. Some people really do think that the world's answers are up on a bookshelf in the Library of Babel, somewhere. The philosophical angle runs deeper yet, though; what kind of cargo-cult society would we build relying on a useful AI? Are we guaranteed meaningful progress because an AI model can keep pressing the "randomize" button? Do we eventually hit a point where fiction and reality are indistinguishable? It's all hard to say.
While that is true, it is also noteworthy that Jensen Huang thinks Tesla is far ahead in self-driving cars. https://autos.yahoo.com/nvidia-ceo-says-tesla-far-110000305....
Tesla still has one local operator per car who has to be able to have twitch reactions at all times.
Competitors like Honda and Mercedes also let you take your hands off the wheel and eyes off the road in certain areas (level 3), which Tesla hasn't yet achieved.
Tesla FSD is still a level 2 system.
I disagree. The day is coming when some *BIG* problem is solved by AI just because someone jokingly asks about it.
I regularly try to ask them to give me fluid dynamics simulation code to see what level they are at. Right now, they can't do that kind of thing all by themselves, and I don't know enough to debug the code they give me.
But even without any questions about free will or consciousness or whatever, a sufficiently capable — not yet existing — transformative search engine (as it has been derided as) and a logical inference engine (which it isn't, but it can use) could have produced the Aclubiere metric with nothing newer than the Einstein field equations and someone asking the right question.
I do not expect transformer models to be good enough to do that given their training requirements, but I wouldn't rule it out either.
The all-encompassing nature of it seems befitting a company producing increasingly general-purpose AI.
It's a doubly linked list where the head contains a pointer to the tail, and a flag that determines which pointer in the nodes is forward and which is backward.
I think it might be similar to game companies where people are attracted to the work itself (whether it’s because they’re True Believers in Musk or because electric cars and space are cool, not sure, probably mostly the latter). This lets the company pay less for the same level of talent, since the work is in itself a form of compensation (as perceived by the people who accept the jobs for lower pay).
Don't get me wrong: Musk has and will continue to get into serious trouble for things he insists are true but nobody else believes (420 etc.), I'm just saying there's a huge gap between them.
The output will very simply tell how much 'truthful' the AI actually is.
We don’t give a s%#* about people wanting to use AI to write SEO spam, automate their customer support or generate content to keep the kids quiet. We want to use this tech as a tool to solve real world problems in a way that, looking back 500 years from now, people will see this as a time of innovation, rather than a time of decline.
Wether he’ll succeed is a different question, of course. But such a direction is clearly missing in the other players. They are just too eager to cater to the laziest segment of the economy of bits. They’re about changing pixels on other people’s screen.