6,603 karma · joined March 13, 2013
TSMC publishes industrial research, not pure science research like a university.
In this context the amount of detail they release is careful balance of PR benefit vs. competitive advantage vs. patent status vs. keeping their talent happy because good researches want their names in prestigious journals.
"On a more philosophical note, we hope that latent reasoning captures facets of human reasoning that defy verbalization, such as spatial thinking, physical intuition or (motor) planning. Over many iterations of the recurrent process, reasoning in a high-dimensional vector space would enable the deep exploration of multiple directions simultaneously, instead of linear thinking, leading to a system capable of exhibiting novel and complex reasoning behavior."
This is a confusing way to describe attention and gets a bit off topic, the attention mechanism is not really what’s causing any of the issues in the article.
I wonder which comparison they hope to avoid, quality or price?
Mine was wanting to make games and hitting the performance wall in basic.
Research papers relating to reasoning approach and define it in many ways but crucially, the good ones offer a testable claim.
Simply saying “models can’t reason” is ambiguous to the point of being unanswerable.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models https://arxiv.org/pdf/2501.19201
It effectively treats “reasoning” as the ability to generate intermediate steps leading to a correct conclusion.
Now, is this valid reasoning? Well, depends on the claim and the definition of reasoning.
When someone just says AI can’t reason, I could argue for or against that depending on the specifics. It’s not enough to just say yes or no.
For example, since reasoning doesn’t yet have any consensus definition that can be applied as a yes/no test - you have to explain what you specifically mean by it, or else the claim is hollow.
Clarify your definition, give a concrete example under that definition of something that’s your version of true scottsman reasoning and something that’s not, then let’s talk.
We know it’s 671B params with each MOE node at 37B…
If the GPUs have say, 140GB for an H200, then do you just load up as many nodes as will fit into a GPU?
How much do interconnects hurt performance vs being able to load the model into a single GPU?
Thanks to Simon for pointing out my point is encapsulated by the AI effect, which also offers an explanation:
"people subconsciously are trying to preserve for themselves some special role in the universe…By discounting artificial intelligence people can continue to feel unique and special.”
I don’t get why people don’t understand that everything decomposes into other things.
You can draw the line for when AI will truly blow your mind anywhere you want, the point is the dominoes keep falling relentlessly and there’s no end in sight.
Therefore, it has no compression ratio, and it doesn’t make sense to compare it to other algorithms.
It doesn’t have a compression ratio.
Heat would definitely kill fusion reactor magnets.
That’s why they’re so hard to build, plasma of a million degrees has to sit only a meters away from superconducting magnets which have to stay cryogenically cooled to near absolute zero.
Say Gemini Flash 8B, allowing ~28 tokens for prompt input at $0.075/1M tokens, plus 2 output tokens at $0.30/1M. Works out to $0.0027 per classification. Or in other words, for 1 penny you could do this classification 3.7 times.
Are any of the side-effect profiles worth it?
It suggests that artificially induced sleep might not provide all the benefits of natural sleep.
How is this not deterministic? Randomness is intentionally added via temperature.
[When you move through space quickly, you move through time slowly]
Why all the “you”? This makes it sound like you would personally experience slowness. Could be clarified to say it’s only according to the perception of others.
Surprised someone was called out though as all the social cues around should be enough to sense no ill intent.
I’m well into this group and still make a lot more api calls than phone calls.
Fresh out of college I recall vividly thinking, I’ll need to build an impressive list of side projects to overcome preconceptions about how much I can truly offer at my age. Maybe nothing has changed.
“Gil’s problem is that the 2019 experiment was long ago superseded anyway: besides the new and more inarguable Google result, IBM, Quantinuum, QuEra, and USTC have now all also reported Random Circuit Sampling experiments with good results.”