fnbr

(I'm a researcher on Olmo.)

There's a bunch of other fully open models, including the [Marin](https://marin.community/) series of models out of Stanford and Nvidia regularly releases fully open models.

MoEs have a lot of technical complexity and aren't well supported in the open source world. We plan to release a MoE soon(ish).

I do think that MoEs are clearly the future. I think we will release more MoEs moving forward once we have the tech in place to do so efficiently. For all use cases except local usage, I think that MoEs are clearly superior to dense models.

(I’m a researcher on the post-training team at Ai2.)

Where did you try this? On the Ai2 playground?

(I’m a researcher on the post-training team at Ai2.)

7B models are mostly useful for local use on consumer GPUs. 32B could be used for a lot of applications. There’s a lot of companies using fine tuned Qwen 3 models that might want to switch to Olmo now that we have released a 32B base model.

Like what? All the examples people have said are where either

1) the company has Nx preferences, for N >1, in which case the company has essentially failed to fundraise or

2) the company sells for less than they raised, which again, is a polite form of failure.

This is why I will never work somewhere with a short post termination exercise period (PTEP). If it’s not at least 5 years, ideally 10, they don’t seriously consider equity something that employees are owed.

Can you explain? In most cases, preferences won’t come into play, assuming you raise at a standard 1x preference and sell for more than you have raised. In that case, owning 0.5% should roughly translate into $5M (modulo dilution).

I hate how every company that I place an order with treats that as permission to send a constant drip of marketing emails. I send them straight to spam.

They raised ~$200M total, and $150M at the series A. So if they're just paying back investors, they'd "only" need $500M.

do we know they were profitable? I doubt it, if they pulled this. I think they had high DAU/MAU but low paid users.

this is basically Inflection 2.0.

Yes, many AI startups pay more than FAANG for ML talent.

What's the dating app?

The rule of thumb is roughly 44gb, as most models are trained in bf16, and require 16 bits per parameter, so 2 bytes. You need a bit more for activations, so maybe 50GB?

you need enough RAM and HBM (GPU RAM) so it’s a constraint on both.

Agreed, that’s what I’m seeing too.

These prices are insane- you can do much, much better than those prices if you negotiate. If you're paying $7.20 per GPU-hour for an H100, you should find another supplier.

Yeah Alberta has very little hydro, while Ontario/Quebec/BC have lots, so our electricity generates a lot more carbon.

That's a great reason not to have electric rental cars, but not a great reason to not charge rental cars.

I meant rent on GCP. You’re right.

I’m surprised they sell any to external customers, to be honest.

Wow. This is surprising. Not even an acquihire. I’m very curious about what happened internally to lead to this.

Private equity firms. I’d never take money from them.

No, unfortunately. But my gas/electricity is fixed price, so it doesn’t matter. I’m sure this will be coming as heat pumps get more common. It’s a pretty easy calculation to do.

Yup, me too. And with my Nest thermostat, I can manually configure the crossover point. I did so at the economic balance point (where the heat pump is cheaper than my gas furnace).

1) i actually think that’s too high, i bet it’s more like 30%. My logic is that they have to have _some_ margin, but LLMs are too expensive to have typical software margins. Total speculation though.

2) It generally tracks pretty well unless the model is gaming the metric (training on the test set, overfit to the specific source of data, etc). The relative rankings will typically match in both.

3) alas, not with the mild winter North America’s having. They only stop below -5C or so. I am lucky though. The woodpecker stopped attacking my house and started attacking my neighbor’s. Even worse, it used to be a downy woodpecker,and it’s now been replaced by a pileated one (think: Woody).

It’s not clear to me what’s happening on the distillation front. I agree no one is doing it externally, but I suspect that the foundation model companies are doing it internally, performance is just too good.

There’s a bunch of recent work that quantizes the activations as well, like fp8-LM. I think that this will come. Quantization support in PyTorch is pretty experimental right now, so I think we’ll see a lot of improvements as it gets better support.

The KV cache piece is tied to the activations imo- once those start getting quantized effectively, the KV cache will follow.

Ha! I got a ton of new subscribers this morning and was wondering why. Let me know if I can answer any questions (I am the author).

Assuming your numbers are correct, they sold for 50x ARR, which is a very very good price. I bet that’s why. It’s hard to do better than that.

Ah, well you could use a standard value network, but it’d end really slow, so you probably want to train a smaller one and rely on the implicit ensembling that MCTS does to make it better.

In my experience, PUCT does a lot better than UCT, so you want to also have a prior network.

You don’t have to train a new network, but in my experience, it works much better. I haven’t spent a ton of time using off the shelf networks with MCTS though. Maybe it works great.

very subtle bugs is the MCTS experience. Particularly once parallelism is involved.

It’s very difficult to implement, and requires training the network to use it.

I worked at DeepMind on projects that used MCTS. Even with access to the AlphaZero source code, it was very difficult to write an other implementation that got the same results as the original.

You need performance in the high level libraries to match, on a flops/$ basis. That’s “it”. That’s easier said than done, though. Even google’s TPUs still struggle to match H100s at flops/$, and they’re really annoying to use unless you’re using Jax.