We don't know what "intelligence" means, but we know it isn't matrix multiplication, or brute force algorithms, otherwise gears could be called intelligent.
I understand the importance of selling. But selling shouldn't be confused with deceiving consumers. It's hard to accept our work is used to create general panic for the sake of money.
We know no such thing. The simplicity of the basic operations do not necessarily constrain the complexity of the whole system composed of those operations. We compose simple operations into complex units all the time.
Through a process called thinking, too.
The same applies to "brute force algorithms" and "otherwise gears could do it".
It's very likely that the current crop of LLMs do the wrong set of matrix multiplications. (If you ask me, it's a certainty.) But that doesn't change the fact that matrix multiplications can do anything.
I didn't, how do you know? And to be honest many LLM seems to show intelligence to some degree, at least when they solve complex and novel problems I ask them in a random language and using N library, that's feel pretty intelligent to me and if it's not than also humans are not.
Neuroscientist here: You're right!
At least in the neuro that we know today, most [0] of the communication between neurons is done in the Fourier domain. In that, it's the frequency of firing events that matters, not that a neuron fired at all.
[0] by no means is it exclusive. The brain is really complicated and there are edge cases all over the place.
Wikipedia is decent-ish on the topic: https://en.wikipedia.org/wiki/Neural_coding
Wulfram Gerstner's book has a nice discussion of the theory and limitations of different proposed coding schemes, which might interest the technical audience here. In particular, I think the parent poster alluded to the ideas presented in the last section here: https://neuronaldynamics.epfl.ch/online/Ch7.S6.html
Really and truly this "discussion" should have been put to rest a long time ago. Neurons encode information with the temporal precision needed to encode that information. Since most objects are somewhat permanent, the temporal precision can be very low, yielding effectively a rate code. For tasks for which the (relative) timing is critical, e.g. the encoding of auditory sequences, the temporal precision of the neural code is high, yielding effectively a temporal code. Some problems are neither here nor there, in which case the neuron will encode as much information as possible at a low rate / temporal resolution, and then encode diminishing amounts of information at successively higher temporal resolutions (the Fourier code alluded to by the parent poster). Higher temporal precision typically requires more control, larger circuits, and hence more energy expenditure.
Bill Bialek's group had a paper that quantified the amount of information present in a visual neuron of the fly at different temporal resolutions (/frequencies) as early as 2008: https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...
I ran into this over ten years ago in an experiment on temporary plasticity of sensory input and I didn't find a quick answer before I had to leave the topic (ok, it left me)
Thanks for sharing that book link, btw.
By this measure "learning" could be inaccurate too. Does a human learn if they just commit something to memory? Programming is chalk full of people who know the code but haven't "learned" to program. Intrinsically we all know there's something deeper to learning than just memorizing or retaining to memory.
I think GP has a point. Marketing terms are to the benefit of the user to be relatable, whether accurate or not. We can and, to my knowledge, historically do retain the stories of what the correlation of a marketing term versus its actual technology is.
Second, the general consensus of what "intelligence" means in the AI field is approximately the Legg&Hutter "Intelligence measures an agent’s ability to achieve goals in a wide range of environments." from https://arxiv.org/abs/0706.3639 .
It's worth noting that solely focuses on the capabilities of the system and intentionally completely disregards how it's built. Gears cannot be called intelligent iff it is impossible to build an system that acts intelligently solely from gears, and if it turns out that it is possible to build a gear arrangement that soundly demonstrates intelligent behavior, then those gears definitely could be called intelligent.
And it's also a clearly nonbinary definition - i.e., it's not whether something is or isn't intelligent, but a scale to measure how intelligent it is; and a observation that a particular system (or human!) demonstrates poor intelligence doesn't prevent us from discussing that it's more intelligent than something or someone who is even worse.
So with unlimited budget and time, can you write something complex enough to seem intelligent? I think so. Is what you wrote actually intelligent? No idea, and I think that’s more philosophy than I’m interested in.
General purpose solving functions will only get better with time and already solve more than we can write solvers for by hand. I don’t suspect there’s a limit here, assuming we can keep improving in ways to scale its compute and scale the function goals.
I believe the post you're replying to is just saying that matrix multiplications (as useful as they are), aren't going to become Skynet.
Your post is pointing out that various AI techniques are replacing loads of jobs for folks that still need to make ends meet, but likely don't have the skill sets to magically become a web developer overnight. As a result, AI is pretty dangerous like many disruptions throughout history. Only in the past, there was usually still plenty of need for labor.
The better we get at understanding how various cognitive functions that once seemed mysterious can be accomplished by matrix multiplications, the easier it becomes to eventually create Skynet.
The weird thing, I think, about these matrix multiplications, seems to be that they might be coming for the jobs of people who are generally in the same field that invented them (programmers) and also they might be coming for the jobs of reporters, creatives, and hot-take authors. People with bigger platforms than factory workers.
What makes you believe “intelligence” can’t emerge from lots of matrix multiplications? Unless you believe in some more mystical explanation, human intelligence is just electrochemical processes not that unlike computers.
Woo boy, someone hasn't taken marketing 101 in uni.