pertymcpert

Why would reallocating resources make a single inference run worse in quality?

Yes but it's their own design, not licensing Arm cores.

It's a custom Fujitsu micro-architecture.

It's not ok if it uses resources that could be better spent elsewhere. At 3% share what do you expect them to do?

I feel like they failed upwards into the right strategy. I doubt it was deliberate and not because of the failure of their foundation models to decent.

> No, Meta just amplified

No. "Just" is understating the effect.

I tested all recent Opus versions and 4.6 and 4.7 were both fine FWIW. Seems 4.8 is when something started to go wrong.

Even React.

Meta employees reading this: whether you work in Reality Labs, ads, operations, ML compiler, LLMs, marketing, whatever. You're all complicit in the harm. Remember that.

You say that like what they were doing was a bad thing. It was a feature.

What processing do they do to the sausage?

> So either the manufacturer-supplied originals are more dangerous than replacement tires

There's 3 main components to tire performance: efficiency, grip and longevity. OEM tires optimize for the first. They can compromise on grip and longevity. Most often I've experienced is the latter. So yes, OEM tires are "worse" in some aspects as replacements.

Most tokens are cached.

Not sure why you had to take a pop at Greggs there. Nothing wrong with a hot sausage roll at all. Assuming you can find a hot one.

You can actually tell from the very first real sentence in the README:

> Efficiency is a 162-run controlled benchmark (same agent, same file tools, only the context differs).

It's so bad that I can tell it's Claude, specifically Opus 4.8/5.0, from the first 6 words.

Right but I've never seen these fp64 computations show up hot in a profile. I'm not saying they're not used but most of the work in a general computer usage in terms of profiles is in scalar integer code.

I really think you should use a more representative benchmark. Heck, even just take Chrome and run some browser benchmark on it like speedometer.

Which workloads?

The issue is theft of IP.

I like how this addresses very little of Apple's core complaints. OpenAI are totally fucked.

You can tell because instead of making an arrow like this: -> it uses a Unicode symbol instead.

Why wouldn’t it?

It's ok, we get it you don't want one. I would happily pay $20k-$30k for one if it worked well. I know a lot of people who would.

I have a 1300 sqft house, 3b2b. Costs me around $200 for a cleaning.

> I'm just wondering. It's OpenAI. Surely there is some domain-specific something beyond "has experience shipping front-end and back-end services" since that includes basically everyone.

Maybe you're looking at this from your own particular point of view too much. I work in systems software, as do many others on HN: our demographic wouldn't be qualified for that at all.

> I'm not sure how this would apply. Are you implying that if the company operates on Python, you can hire someone with great "raw intelligence" who have only developed C++ all their life, and they can start contributing on day 1?

In your particular case, no not on day 1. But they can probably make meaningful contributions after 2 weeks. They'll pick it up quick because they're some of the best talent on the market.

The applicability of your advice is very model dependent. Some like Claude have very good long context performance, whereas others they fall off much quicker past some threshold.

It's pretty painful to have speeds < 30 tok/sec though. Especially if you're used to API providers at higher speeds. It makes any interactive work almost impossible to do efficiently because you have no choice but to context switch after every request.

If you didn't already know, jobs at highly competitive companies tend to have vague job requirements because they expect to be able to apply your raw intelligence to changing demands quickly. There's no point being hyper-specific about the exact software packages because that's not what they want. What they want is someone who, after talking to an interviewer for 30 minutes, leaves them with the thought "Wow, this person can do anything we need of them. They can probably tell us what we need too and take ownership of large projects. Hire!"

Well put, great article.