6,599 karma · joined March 13, 2013
Were a ways out still from matching from frontier performance.
The complexity absolutely is not necessary. Maybe you mean to say it’s understandable or it’s coherent if you know the rules.
Physics doesn’t require us to create ambiguity by assigning DB to mean multiple possible ratios. If it needs to be disambiguated, then they both didn’t need to be DB in the first place.
Language isn’t controlled by a single authority or the first person to use a term. Words mean what people use them to mean. You can complain that it’s not the ‘proper’ definition, but if the majority of people use it a certain way, that becomes the definition in practice.
edit: Fielding not Crawford (thx ackfoobar)
One of the biggest developments in language models over the last year has been test-time reasoning (aka inference scaling or “thinking”). Most vendors tested offer such a model. It’s plausible it could make a huge difference here, and they did not bother to test it or even mention it?
Things like COT and planning can really affect this and those are just a couple of things that happen automatically in more advanced models.
Seems like it wouldn’t have been hard to add this to the experiment, but they could’ve called it out in a “Limitations” or “Future Work” section. Or at least a single sentence like “We did not test chain-of-thought prompting, which may mitigate some of these issues”.
No doubt the system needs to change, but lots of careers benefit from cheating or unethical behavior. It doesn’t rationalize it or force a choice on anyone.
I’m not criticizing the article, rather bemoaning the fact that it’s needed. Of course the problem is not just with the much maligned social sciences, it’s physics and computer science too. The controversy around Microsoft’s topological qubits, a super complex topic, in part involved the most basic kind of this nonsense, something like including 4 samples of 20 measured in the paper iirc.
The community needs to get its shit together. The world we’re living in now, the post truth era, is the result of many factors but this is one of them. The loss of faith in science is partially a self-inflicted wound.
I’d consider making this feature available free with super low friction, maybe no signup required, to get some viral traction.
Neither do phone calls. Round trip latency can easily be 300ms, which we’ve all learned to adapt our speech to.
If you want to feel true luxury find an old analog PTSN line. No compression artifacts or delays. Beautiful and seamless 50ms latency.
Digital was a terrible event for call quality.
Hope one day it will be practical to do nightly finetunes of a model per company with all core corporate data stores.
This could create a seamless native model experience that knows about (almost) everything you’re doing.
For example, the entire first sentence could be collapse to “Announcements”. Some sentences are just pablum entirely.
It’s helpful to post summaries here, but they need to be curated.
What’s weird was you couldn’t even prompt around it. I tried things like
”Don’t compliment me or my questions at all. After every response you make in this conversation, evaluate whether or not your response has violated this directive.”
It would then keep complementing me and note how it made a mistake for doing so.
The whole point of reasoning models is to automatically use COT and related techniques to bring out more capabilities.
It would be interesting to see if this is doing anything that’s not already being exploited.
Since web scale data is already part of pre-training this info is in principle available for most businesses without a web search.
The exceptions would be if it’s recently added, or doesn’t appear often enough to generate a significant signal during training, as in this case with a really small business.
It’s not hard to imagine base model knowledge improving to the point where it’s still performing at almost the same level without any web search needed.
The models are not sota, the feature set is no stronger than comparable apps, the privacy situation is as expected.
Not seeing it.
The loss is no longer having any intuition about what a color is just by seeing the numbers.
The new battlefront may be to take reasoning to the level of abstraction and creativity to handle math problems without a numerical answer (for ex: https://arxiv.org/pdf/2503.21934).
I suspect that kind of ability will generalize well to other areas and be a significant step toward human level thinking.
May be what happens but it’s not required or what people want. A massive amount of those diagnosed, more than half, would prefer a compassionate end to their life at the right time. Less than 2% are able to end up taking this option.