make3

The output key is determined through code by the harness from the inputs, and the model generates 255 floats in order that follow that schema by reading the expected return type "rank" in the input. The harness then programmatically uses the floats of the 255 floats that are useful. The harness can assume that the correct float will be in the correct position as the model is trained to follow the schemas

no, 255 numbers come out at once all the time, the order is determined by the inputs

logits assumed some form of softmax or logistic, which may not be the case

FLAN-T5 generated text (Jev does not generate text), and BERT wasn't able to do tasks without fine-tuning.

Jev is basically a kind of FLAN-BERT, if you want, where it has built-in multi-task ability, but doesn't generate text. It only generates 255 floats all at once, making it much faster, and what those floats mean (if anything) depends on the prompt.

Eg, the following query is put in the encoder model:

{"question": "Rank these 5 things by increasing order of how big they are", "choices": ["truck", "cow", "mouse", "ant", "building"] }

The model returns [3., 2., 1., 0., 4.], and 249 other meaningless floats that are hidden from you by the UI.

The UI stitches the first 5 floats with the choices and returns something like:

{"rank": ["ant", "mouse", "cow", "truck", "tower"]}

the appeal would be if they can deliver it at a much higher performance and similar speed, which is plausible

Your joke also shows why it's dumb though.

We have general AI capable of proving professional mathematician level math theorems and capable of taking down global infrastructure through hacking and that completely changed what it means to be a software engineer, but we're evaluating it by generating fucking SVGs for the same animal in the same conditions, and it's supposed to mean something somehow

they're is obviously joking

This benchmark is useless and should die. LLMs have likely trained on it, it's too easy to game by training specifically for it, & it doesn't mean much

parent means that they could get more client / a larger part of the market, which would lead to more income (more tokens) despite lower marginal prices

> Unfortunately, this might be an example of OpenAI the company having vastly more compute time and effort than your average enigma nerd.

I don't think this applies, isn't this just the researcher using ChatGPT Codex on their machine?

distillation still costs quite a bit, even if a lot less. "quite a bit" could mean millions. it's also not very discreet for the model to generate the millions and millions and millions of tokens to be distilled, vs uploading a few terabytes

>Society profits from having AI

Not all of society at all. Let's discuss this when AI is better integrated and a large fraction of people are laid off in 5 years.

Imagine if they trigger the California Big One and all of LA, San Diego and the SF Bay (and innumerable towns) are swallowed "because of man-made actions", I wouldn't want to be the Governor or President to approve that one lol

Frontier model trainers stole almost all the data they've trained on (the whole internet, all copyrighted).

It's very hard for them to claim the moral high ground here.

It's like stealing an apple from the British Colonial Empire.

shaming people for bad security practices is probably net good, whether we like it or not

honnestly I would be less surprised if a human learned the info and prompted the model in the right direction

Then your choice is either a US mega corp under the Trump partial dictatorship shitshow or a Chinese model distillation factory that could turn on you at any second.

I understand that there are users that would prefer another option, especially in the EU.

Brain dead moral relativism argument. The question is whether eg. a group trying to scam elders out of insurance money or a Columbian cartel to hack local politicians to do blackmail, or South Sudan to hack Darfur or whatever, should be allowed to compete with the companies making products for their own exploits.

99.999% of people will agree that reducing software vulnerabilities is desirable if they're able to understand the question, including the bad actors themselves a lot of the times.

The situations like bad state actors are already not bound by laws, and things like keeping activism legal are better fought for through other ways

Lean is adversarial in a way. Lean is better thought of as a constraint language with a verifier that checks if the constraints are respected, than a programming language.

Your job or the LLM's job is to write code that Lean is satisfied with, creating the link between what you're trying to prove, and mathematical axioms.

If you write a bad proof, the Lean constraint checker will tell you, unless there are bugs in Lean itself, or you defined the goal constraint incorrectly.

Interesting. Indeed, proving theorems that are stronger and more general "accidentally" than what you really need is not a bad thing.

you would never assume this if you've spoken to any human being, ever

Doing things the hard way is a lot of times "male" bravado (though this is not limited to males in any way ofc), this is a very common thing in junior engineers.

Curiosity is good but maturity is trusting that the problem will get hard at some point anyways, that improving the most efficient way is usually also interesting and is more likely to deliver desired things on time or at all, which can be a big deal if what you're trying to deliver it worth it, like.. a new MRI machine that detects new types of cancers, etc.

If you don't care about what you're trying to deliver, that's another problem that requires at least some questioning, though I understand people have families to take care of etc.

surely you'll get sued by the GLP-1 patent owners like HIMs did if you sell your own product? & can physicians recommend non-FDA approved manufactured drugs?

My assumption is that they're cooking an ultra humongous Gemini 4 Pro release. They certainly have the cash and the compute for it, and it's so obviously the thing to do from a strategic perspective.

That extremely likely just means that they're preparing an omega huge Gemini 4 Pro release and that that's what training right now on most of the compute

brother H-1Bs are transparently vectors for foreign educated workers to come to the US to build lives and eventually get a green card, so the US doesn't have to actually build a functional education system.

no one talented will come pay taxes and make your nice tech companies carry the S&P500 and in turn, everyone's retirement funds, if you treat them like hogs

NVidia has reasonable incentives to keep things open indefinitely, it wants people to use it's GPUs

I think this is good.. They have incentives to keep things free to keep people using their GPUs. I think that's one of the least enshittifying outcomes possible

& the town square forces you to listen to famous gossippers and dumb popular people you've never heard of

they mention using an open weights model, which one it is doesn't really matter