>> Truly open source coming from China.
> Open weight!
They clearly were implying it's not open source.
11,355 karma · joined August 11, 2008
> Open weight!
They clearly were implying it's not open source.
Did you even read my comment?
And the active parameters come from the experts. For each token the model picks some experts to run the pass (usually 2 to 4, I haven't read V4's papers). It's not always the same experts.
OTOH, being DeepSeek, I foresee a bunch of V4 distilled FP8 models fitting in a 5090 with tiny batches and with performance close from 75 to 85% of V4. And this might be good enough for many everyday tasks.
Today is a good day for open models. Thank god for DeepSeek.
Just this week they published a serious foundational library for LLMs https://github.com/deepseek-ai/TileKernels
Others worth mentioning:
https://github.com/deepseek-ai/DeepGEMM a competitive foundational library
https://github.com/deepseek-ai/Engram
https://github.com/deepseek-ai/DeepSeek-V3
https://github.com/deepseek-ai/DeepSeek-R1
https://github.com/deepseek-ai/DeepSeek-OCR-2
They have 33 repos and counting: https://github.com/orgs/deepseek-ai/repositories?type=all
And DeepSeek often has very cool new approaches to AI copied by the rest. Many others copied their tech. And some of those have 10x or 100x the GPU training budget and that's their moat to stay competitive.
The models from Chinese Big Tech and some of the small ones are open weights only. (and allegedly benchmaxxed) (see https://xcancel.com/N8Programs/status/2044408755790508113). Not the same.
"Qwen's base models live in a very exam-heavy basin - distinct from other base models like llama/gemma. Shown below are the embeddings from randomly sampled rollouts from ambiguous initial words like "The" and "A":"
It's just speculative decoding but for training. If they did at this scale it's quite an achievement because training is very fragile when doing these kinds of tricks.
They are definitely distilling it into a much smaller model and ~98% as good, like everybody does.
I really wish AMD and Intel boards get replaced by competent people. They could do it in very short time. Both have integrated GPUs with main memory. AMD and Intel have (or at least used to have) serious know-how in data buses and interconnects, respectively. But I don't see any of that happening.
ROCm? It can't even support decent Attention. It lacks a lot of features and NVIDIA is adding more each year. Soon they will reach escape velocity and nobody will catch them for a decade. smh
Indeed. Anthropic is just leading the pack switching to juicy corporate users who are happy to pay thousands per month per dev and leave the fans behind. And now OpenAI is following suit. They lowered significantly the limits for the Plus $20 plan and answered concerns with vague confusing tweets about promotions.
All this is pushed by the fastest rising demand (Codex growing +50% monthly) while having a serious bottleneck building data centers and getting parts (permits, energy, memory, flash, etc).
Users on reddit and Discord are trying to switch to open models or Chinese alternatives. But there's no real replacement.
So long, Hollywood.
Boomers already started to burn their $78 trillions in savings. And taxes will skyrocket for the rest to pay their fat unfunded pensions. Oh, and don't forget giving them subsidized/free healthcare. And a last FU, they collude to rise rent (they are the landlord class).
But hey, they never forget to vote.
A matter of perspective.
As it's done in most of Europe, Latin America, Asia, Africa. US/Canada were turned against family and tradition thanks to decades of brainwashing by movies, TV, and ads. To be cool you had to go to a college in another state. Then move to a big city. Pick career over family and having children. Take antidepressants.
This started in the 70s and is well documented.
Middle East war to push American price growth to ‘highest in G7’
The Middle East crisis will fuel a surge in US inflation to 4.2 per cent this year, the highest in the G7, according to an OECD forecast that highlights the cost of the US-Israeli war with Iran.
* Always take sales people's stories with a grain of salt
Exclusive: OpenAI sweetens private equity pitch amid enterprise turf war with Anthropic, sources say
NEW YORK, March 23 (Reuters) - ChatGPT maker OpenAI is offering private-equity firms a sweeter deal than rival Anthropic as both artificial intelligence companies court buyout firms to form joint ventures aimed at raising fresh capital and accelerating adoption of enterprise AI products, according to people familiar with the talks.
OpenAI is offering private-equity firms a guaranteed minimum return of 17.5%, significantly higher than typical preferred instruments, two people familiar said. It is also offering early access to its newest AI models as it seeks to enlist investors like TPG and Advent for its joint venture, three sources said.
I’m not American, and yet I can tell this is extremely unlikely to happen. I’ve lost track of all the Cassandras raising the alarm who were shot down or even laughed at. But there is still a bit more road left for the can to be kicked one more time.
But the take from this post sounds ridiculous. I clicked expecting some take on consistency and some nice paradigm. But no, some weird-ass ranting with 80s UIs. I couldn't keep reading.
There are Crypto perpetual futures markets open all weekend but those are much smaller so it's unlikely they could make millions there.
And Brent futures are starting to climb up again fast.
Trump's post is either him freaking out of a Black Monday and/or losing military command, or it can be interpreted as market manipulation. But I don't think he is much in control anymore.
Serious oil traders are saying it will take months or even more than a year to get back to normal.
The only thing containing prices at the moment is many exporters sold futures to lock-in prices for the rest of the year (it wasn't market manipulation as many suspected). But once they are sold out we'll have some interesting price discovery.
https://www.tradingview.com/symbols/ICEEUR-BRN1!/forward-cur...
https://www.tradingview.com/symbols/NYMEX-CL1!/forward-curve...
Oil Market’s Seaborne Buffer Runs Down Fast as Iran War Drags On