Random observation 2: It's time to cancel the OpenAI subscription.
Don’t get me wrong what DS did is great, but anyone thinking this reshape the fundamental trend of scaling laws and make compute irrelevant is dead wrong. I’m sure OpenAI doesn’t really enjoy the PR right now, but guess what OpenAI/Google/Meta/Anthropic can do if you give them a recipe for 11x more efficient training ? They can scale it to their 100k GPUs clusters and still blow everything. This will be textbook Jevons paradox.
Compute is still king and OpenAI has worked on their training platform longer than anyone.
Of course as soon as the next best model is released, we can train on its output and catch up at a fraction of the cost, and thus the infinite bunny hopping will continue.
But OpenAI is very much alive.
This entire hype cycle has long been completely disconnected from reality. I've watched a lot of hype waves, and I've never seen one that oscillates so wildly.
I think you're right that OpenAI isn't as hurt by DeepSeek as the mass panic would lead one to believe, but it's also true that DeepSeek exposes how blown out of proportion the initial hype waves were and how inflated the valuations are for this tech.
Meta has been demonstrating for a while that models are a commodity, not a product you can build a business on. DeepSeek proves that conclusively. OpenAI isn't finished, but they need to continue down the path they've already started and give up the idea that "getting to AGI" is a business model that doesn't require them to think about product.
Couldn’t agree more! Nobody here read the manual. The last paragraph of DeepSeek’s R1 paper:
> Software Engineering Tasks: Due to the long evaluation times, which impact the efficiency of the RL process, large-scale RL has not been applied extensively in software engineering tasks. As a result, DeepSeek-R1 has not demonstrated a huge improvement over DeepSeek-V3 on software engineering benchmarks. Future versions will address this by implementing rejection sampling on software engineering data or incorporating asynchronous evaluations during the RL process to improve efficiency.
Just based on my evaluations so far, R1 is not even an improvement on V3 in terms of real world coding problems because it gets stuck in stupid reasoning loops like whether “write C++ code to …” means it can use a C library or has to find a C++ wrapper which doesn’t exist.
OpenAI doesn't have an advantage in compute more than Google, Microsoft or someone with a few billions of $$.
Why would anyone bet? They can just short the OpenAI / MS stocks, and see in a few months if they were right or not.
Need an LLM to one-shot some complex network scripting? as of last night, o1 is still where its at.
Alex (https://alexcodes.app) also does this now btw.
The benchmark comparisons are perhaps, for now, the best way to compare reasoning prowess of R1 vs O1, since it seems pretty certain they both trained for those cases.
I think the real significance of R1 isn't the released model/weights itself, but more the paper detailing (sans training data) how to replicate it, and how effective "distillation" (i.e. generate synthetic reasoning data for SFT) can be to enhance reasoning even without using RL.
Of course cost is incomparably higher since plus has a very low limit. Which of course is a huge deal.
O1 vs R1 performance on specific non-benchmark problems is also not that relevant until people have replicated R1 and/or tried fine-tuning it with additional data. What would be interesting to see is whether (given the different usage of RL) there is any difference in how well R1 vs O1 generalize to reasoning capability over domains they were not specifically trained for. I'd expect that neither do that well, but not knowing details of what they were trained on makes it hard to test.
2. If you have GitHub Copilot, you get o1 chat also there.
I haven't seen much value with OpenAI subscription for ages.
Deepinfra is pretty cheap though as a deepseek provider.
As for deepseek, I couldn't even sign up because my email domain is not on their whitelist. To just try it out for now I don't mind the increased cost.
ChatGPT is the king of the multimodal experience still. Anthropic is a distant second, only because it lets you upload images from the clipboard and responds to them, but it can't do anything else like generate images - sometimes it will do a flowchat which is kind of cool, GPT won't do that - but will it speak to you, have tones, listen to you? no.
And in the open source side, this area has been stagnant for like 18 months. There is no cohesive multimodal experience yet. Just a couple vision models with chat capabilities and pretty pathetic GUIs to support them. You have to still do everything yourself there.
There is a huge utility for me, and many others that dont know it yet, if we could just load a couple models at once that work together seamlessly in a single seamless GUI like how ChatGPT works.
AFAIK you can't do that with newer consumer cards, which is why this became an annoyance. Even a RTX 4070 Ti with its 12 GB would be fine, if you could easily stack a bunch of them like you used to be able with older cards.
I’d guess they didn’t quite a bit of fine tuning to censor some more sensitive topics which probably impacts the output quality for other non technical subjects.
The people making the model probably don't really give a shit about politics and just did the minimum to avoid being embarassed, but if people start jailbreaking it they will be forced to care.
I don't give a damn about ideology I just want everything ever thought or written searchable and interactive
How many laptops have you personally seen be stolen on a train?
Depending on the train type and configuration, many actually seem like pickpocket paradise.
FWIW I’ve used my laptop on the train plenty, I’ve never had anything stolen nor felt in any danger of it.
So a little trick I figured out is to close my laptop lid and then slide it into a pocket of my backpack. I can then carry it with me when I get up and move around.
So then I can take it with me to eat lunch or an extended toilet break. Maybe some day all laptops will have that feature.
People get up to use the bathroom or the cafe car, the laptop is left behind for ten minutes, one of the train stops is while they're away from their seat, and someone sees an opportunity, snags it, and gets off at the stop.
This is an actual thing. And if it's worth a thousand bucks then it's very much worth getting off at an earlier stop then you'd planned, and continuing your journey on the next train.
Ticket inspectors or guards are irrelevant. There isn't one in your car 99% of the time.
I don't why you're trying to argue laptop theft on trains in first-world countries isn't a thing. It absolutely is.
So, yes, theft on trains for people that think they are 100% safe are a thing, but applying the same idea (to assume something is 100% safe and not be cautious) I wonder how do such people use the internet...
The attempted thief didn't succeed in taking the phone, but did knock the laptop onto the ground, damaging it.
If I would make a statistics of primary cause of remaining without a laptop among people I know, the biggest danger is liquids in glasses (that ends up on the laptops) ...
I don't think I've ever seen a human being do that before on a train. Not to go to the toilet, nor to grab a coffee in another car.
You can't be paranoid about everything. My friend in France had put his laptop back into his bag where it wasn't visible and assumed that was good enough, but someone must have seen him do it and just took the whole bag.
You are applying a totally unreasonable standard, to suppose that the thefts were due to unreasonable carelessness. What, do you think someone should take their large luggage into the bathroom too, every time they need to pee?
Talk about victim-blaming.
The standard is mine and I follow it. The same way I find absurd not to do it, you find it unreasonable to do it.
I find the expectation that things are not stolen (if unsupervised in public places) strange considering the huge amount of inequalities in wealth around even in civilized countries. I do not agree with the idea of stealing, thiefs should be punished, but expecting everybody "to behave" given the situation seems unrealistic to me.
That does not mean that I think that things are stolen 100% of the time. I have a friend that forgot a laptop on a bus (Netherlands) and the driver found it at the end of the line and gave it to lost objects so my friend got it back.
If you find it absurd how 99% of people act on long-distance trains, I don't know what to tell you.
My work policies state you simply cannot leave your laptop out of sight for any period unless it's in a secure location (work|home). I feel the same way for my personal laptop as well.
Obviously, nobody steals things while the train is in motion. They wait until the train is about to leave the station, snatch a phone or handbag and jump out just as the door is closing. The train leaves, the thief blends in with other passenger leaving the station, and by the time news of the theft has made it from the passengers to the driver to the station staff the thief is long gone.
Of course people drive around $6,000+ cars all the time, so....
Something interesting: I live near a train line where the doors are not automatic (they have to be opened manually on each stop), and there have been incidents where people get pickpocketed while the train is still in motion, and the thief jumps out right before the station, when the train has slowed down significantly but is still in motion. Many people have been hurt doing this.
Make sure the laptop is insured and that full disk encryption is enabled. If it’s a Mac, make sure you have it in Find My so you can wipe it remotely if that’s something you worry about.
Today's baseline laptops are really good as it is. 32-64 GiB of RAM is plenty, and at least on PC laptops you can do it fairly cheaply. Apple has been a consistent year or two ahead in mobile CPU performance but it fell out of my consideration ever since I realized the M1 and 7040 were both very sufficient for any local computation I cared about. (I'm not going to say I'd specifically go for less efficiency or performance, but it has become significantly lower priority over other things like repairability.)
Not really specifically hating on Apple, here. If I was going to get another Mac it'd be a Mac Mini or Mac Studio probably, ideally with a third-party SSD upgrade to both save on costs and get a slight bit of extra drive performance too. I've definitely considered it, even though I am very far from an Apple fan, just due to the superior value and efficiency they have in many categories.
Yesterday's entry: "... kind of a mind flex that you noted you used Meta Stories glasses to take that photo."
But yes, you're right. I've never personally seen a laptop get stolen. In fact, most people who have their laptop get stolen never see their laptop get stolen either.
I have, however, had coworkers who've had their laptops stolen. Multiple times.
What kind of timescale do you expect to be able to train a useful LLM with that?
That's because it's Apple. It time to start moving to AMD systems with shared memory. My Zen 3 APU system has 64GB these days and its a mini ITX board.
It's better to get (VRAM + RAM) >= 140GB for at least 30 to 40 tokens/s, and if VRAM >= 140GB, then it can approach 140 tokens/s!
Another trick is to accept more than 8 experts per pass - it'll be slower, but might be more accurate. You could even try reducing the # of experts to say 6 or 7 for low FLOP machines!
Can you release slightly bigger quant versions? Would enjoy something that runs well on 8x32 v100 and 8x80 A100.
Apple's M chips, AMD's Strix Point/Halo chips, Intel's Arc iGPUs, Nvidia's Jetsons. The main issue with all of these though is the lack of raw compute to complement the ability to load insanely large models.
It seems that AMD Epyc CPUs support terabytes of ram, some are as cheap as 1000 EUR. why not just run the full R1 model on that - seems that it would be much cheaper than multiple of those insane NVidia-Karten.
I guess the 5090 either started ever so slightly to become compute limited as well, or hit some overhead limitation.
[1]: https://www.phoronix.com/review/nvidia-rtx5090-llama-cpp
>Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage.
Even if $3k is only the starting price, it doesn't sound like spending more buys you more memory.
But everyone is using the distilled models which are much smaller.