So much of neurodegeneration has turned out to be the effects of latent viruses in the body. HSV-1 is correlated with developing dementia along with many respiratory viruses.
4,908 karma · joined February 23, 2023
> No, it can hold more floating point numbers.
Fallacy of composition. Just because an LLM is made up of floating point numbers doesn't mean its capabilities are limited to that of bare floating point numbers, in the same way that the individual faculties of a neuron don't preclude the human brain from emergent properties born from the synthesis of its synapses.
He claims that his company has "solved" hallucination by creating a verifiable fact-finding system, which is like saying that a person has solved plan crashes by creating a plane that never leaves the ground.
When an LLM says something incorrect, it often is due to that LLM reaching the limits of its abilities, but it doesn't "know" (for lack of a better term) what being wrong feels like, so it will try its best to fit the information it has into a compelling story. The reason why scaling leads to fewer hallucinations is that the model can hold more abstractions, more facts about the world, it can work through the complex, vague machinery of reason with more scaffolding, and more of a buffer (via its weights) to reason with nuance. This is why LLM's are useful, not because they can be fed into a fact-retrieval system, but because they can produce new information via the association of things they know.
The point is, we want LLM's to actually produce new information and work out things via their thinking, not be limited to citing facts that already exist and avoid veering into the limits of its abilities. In that sense hallucination is really just exposing the limits of scale, which would necessitate scaling models further.
Scaling is the only way we have gotten to this interesting, emergent property of LLM's. Further, the best way to make small models which don't hallucinate (that we've found so far) is to train a big model first, then distill it, or use it as a teacher to a smaller model. Either way, pursuing scale is the most defensible strategy, and a more robust solution to hallucination.
Want frontier intelligence? Better not defy the current administration, or your competitors will have access to a better model you could never use.
Like lets say i gave a human and an LLM a tech spec and the LLM created a software product as good as the human, does it matter that there was consciousness behind the human thinking?
What is your definition of thinking?
AI is good enough now that you can't claim that you aren't using it because you're upholding some higher standard of quality. It is simply a matter of it offending your sensibilities.
RAM prices were going to go up no matter what given the demand for AI. To so strongly want someone to suffer for fulfilling his fiduciary responsibility to the company he runs seems a bit disproportionate.
> It is undeniable that the entire AI sphere is a bubble, artificially propped up by circular investments and wash trades, and that is now poised to raid pension funds.
I honestly can't take your claims that AI is a huge artificial bubble seriously when you make it very clear how much you want it to be a bubble. The mark of a mature mind is being able to admit that people you personally dislike actually may be right about a few things.
That these people whom you hate so much are also cartoon villains who have taken the entire world economy for a ride is a very satisfying narrative because it implies a future point when all of their crimes will be revealed and they will be held accountable. It is however not a realistic thing to hope for. This is all because of one very real but hard to admit reality: AI is a very useful technology and is worth the billions of dollars that are being paid for it. To deny this reality at this point, when AI models have already proven unsolved Erdos problems and are driving such huge demand on both the consumer and enterprise side, is unreasonable, and consequentially smart people who would otherwise take your points seriously are going to see right through your claims as just being emotional self-soothing.
1. Benchmark performance of LLM's and AI models did not fully represent skill in real-world domains
2. Most jobs span far more requirements than their specific job descriptions, many of which lie, even in simple jobs, in the realm of highly adaptive, context rich multi-modal information processing that most humans do still better than AI
However, there is nothing fundamental that prevents LLM's from scaling and improving, aided by better scaffolding, to the point of replacing many white-collar jobs, especially ones which have limited, specific requirements and output parameters. This is an enormous chunk of the white collar work force, and displacement is already happening in limited sections, and will surely continue as AI capabilities diffuse.
It seems however deeply entrenched in many people's identity to deny this fact, because to accept it requires accepting that many of the essential claims of AI CEO's are somewhat true to a degree, and that LLM's are a genuinely useful technology.
I would happily read an AI-critical blogpost if it weren't clearly motivated by a strange, specific hatred of the prominent AI figureheads.
At this point I automatically dismiss writing like this, the motivated reasoning is palpable. Their distaste for the character and general vibes of the AI industry trap them in blatant denials of reality, like claiming that AI is a completely worthless technology or surely the bubble will pop any minute now.
I am all for well-researched criticisms of these companies and their claims, but please start at the facts and use them to derive your conclusions, rather than the other way around.
The essence of fair use is that it is ok if the use of the copyrighted material is transformative. LLM training I believe is transformative, as it is taking the data (as in a work of text, an image, video), and feeding it through the layers of a neural net to marginally update its weights. It is factoring each work into a very very small fraction of the model's overall sense of the world.
Now, AI models are capable of reproducing copyrighted works to a degree of high accuracy, especially if those works recur very frequently in the training data, but I believe that's a different issue. Any video camera is capable of taking a photo of a copyrighted work, but that isn't essential to the value of the camera, though it is undeniably what certain cameras are used for. The exact reproduction of copyrighted works is a likewise something that LLM's can do, as any intelligent person could recite song lyrics or a work if they memorized it, but each individual work is only marginal to the overall effectiveness and value proposition of the model.
I can't really trust any anti-AI argument when it feels more of a tribal grievance than a rational explanation of concern. Especially with the overuse of the "techbro" pejorative, it seems more a lament against a certain type of attidude in the tech world and a hatred that that attitude has translated into massive material wealth.
The whole point of the company is that they are investing a huge amount of money upfront in order to make models that are better and better, and thus have a higher productivity multiplier.
They are very profitable on inference, they just know that the race to AGI requires a huge amount of investment, compute, getting the best researchers, etc.
>It is not ok to lobby and manipulate governments for fiscal policy changes to write off the costs and risks on the general public.
How are they doing this? What are you referring to?
>The tech industry doesn't speak for the entire planet
Of course they don't, but they are still allowed to make a product, and pivot if there's no consumer demand. However there is huge consumer and business demand for AI so they are justified in making the investments they are.
The economy is human-serving and human-legible because humans are a universal bottleneck to productivity and wealth creation. Once humans are no longer this bottleneck, the economy will begin to serve other interests, as those other interests will be the source of far more productivity than humans (i.e. AI agents, robots, etc)
If people understood this they'd understand that the "permanent underclass" notion is farcical: Human capital will not be allowed to be what allows productivity to commence or halt in a future that is 1000x more efficient and fast-moving due to AI. Any AI smart enough to do such will not wait on humans to give them permission with their money.
As long as the models keep getting better, you will keep losing betting against them.