11,713 karma · joined May 13, 2015
Given the recent deepseekv4.1 advances - how good of a 3B model can we make to run on an iphone natively? is it good enough to match common muse/dot use cases for consumers? the phone is already always on.. no need for a cloud server.
While I’m fortunate now to be able to dedicate time to the gym and other physical fitness, this was not always the case.
We lack the societal incentive structure to allow someone working 60-80 hours a week to reduce their workload to physically recover. We do allow that same person to spend 5-10 years income on surgeries which have 4-6 week recovery times.
Something like abstractions traversed during interpretation, lines of abstraction v.s. functional implementation, or logic statement dispersion.
It was hard to pin down what was abstraction vs. implementation, but it's much easier now.
Missing organs, odd tissue performing unclear functions, different, new or missing muscles.
It’s unclear whether a world ending virus is even possible. There might be better luck with prions, or fungi.
Even if all GCR models were banned, anyone with access to 20k-1MM GPUs can throw their hat in the ring to make a new frontier open model.
If demand vanishes for the 3 million cards Amazon just bought, then something will be done with them. The AI market may end up in a bizarre jepson's paradox of rotation between inference use cases and model training.
Minimally. It could say that no such analytic function can ever exist. Which would be rather boring.
Turbulent fluids look awfully predictable with their spirals….
1 year ago we viewed models as tools and agents were just kinda toying around, that we now think the bar is literally an anything to anything converter through one agent is wild.
Is it rapid skill acquisition? -> ARC benchmarks are saturated Is it breadth of knowledge? -> See many ... many benchmarks Is it ability to do hard tasks? -> see terminal-bench and released outputs.
We are at the point where the starting point for most tasks should be "send your agent to work on it."
So where do we draw the line in a way that doesn't move every 6 months?
We overpay for healthcare because we must pay, and it has institutional and natural bottlenecks which restrict availability and choice.
A big enterprise can drop one (or 4) of these on someone's desk and let them go nuts.
if they are a heavy user, perhaps they string 4x together.
The advent of factories with limitless and constant demand for labor is a new development of the industrial revolution.
At the investment scales being discussed, CUDA/architecture and other advantages do not matter - you could spend 1 billion on building a new chip architecture. The ram/fab inputs have been a commodity market for years. Heck, even the model bottleneck doesn't seem real when it's only 1-4 billion or less to get a state of the art model.
At some point the compute bottleneck will be relieved, you can see NVidia hedging their strategy with both open models and on-device chips targeted for local inference. The 200 dollar a month plan will absolutely be taken over by local hardware in the future.
If a time traveler went back to 1600 and started spouting off about differential equations everyone would think them quite mad.
Take that data / training recipe and scale it up with compute to see what happens?