chaos_emergent

the point is that minimizing 4 months as an insufficient timeline along which to prove out ones ability to predict a sequence of events is dumb when the rate of progress is incredibly fast.

The same technology that checks notes has been in the wild for 7 months?

The idea of chatbots existed before ChatGPT, does that mean it's purely marketing hype?

I disagree that there isn't an innovation.

The technology for reasoning models is the ability to do RL on verifiable tasks, with the some (as-of-yet unpublished, but well-known) search over reasoning chains, with a (presumably neural) reasoning fragment proposal machine, and a (presumably neural) scoring machine for those reasoning fragments.

The technology for agents is effectively the same, with some currently-in-R&D way to scale the training architecture for longer-horizon tasks. ChatGPT agent or o3/o4-mini are likely the first published models that take advantage of this research.

It's fairly obvious that this is the direction that all the AI labs are going if you go to SF house parties or listen to AI insiders like Dwarkesh Patel.

for the longest time I envisioned some sort of configuration specification that could retrieve URLs, transform and map data, handle complex conditional flows...and then I realized that I wanted a Normal Programming Language for Commerce and started asking o3 to write me Python scripts.

as I sit in front of my computer waiting for a transformation-for-import job to complete, I can describe my basic workflow:

1. define a clean interface target - for me, that's an interface that I made for my startup to import call data.

2. explore the data a little to get a sense of transformation mappings.

3. create a PII-redacted version of the file, upload it to ChatGPT along with the shape of my interface, ask it to write a transformation script in Python

4. run it on a subset of the data locally to verify that it works.

5. run it in production against my customer's account.

I'm curious - that seems like a reasonably standard flow, and it involves a bit of manual work, but it seems like the right tradeoff between toil and automation. Do you struggle with that workflow or think it could be better somehow?

he didn't step down, he just wanted to code instead of being a manager.

If it smells like doomerism click-bait...

Theory here is that 4.1-nano is competing with that tier, 4.1 with flash-thinking (although likely to do significantly worse), and o4-mini or o3-large will compete with 2.5 thinking

based on their release cadence, I suspect that o4-mini will compete on price, performance, and context length with the rest of these models.

I meant this is actually straight-forward if you've been paying even the remotest of attention.

Chronologically:

GPT-4, GPT-4 Turbo, GPT-4o, o1-preview/o1-mini, o1/o3-mini/o3-mini-high/o1-pro, gpt-4.5, gpt-4.1

Model iterations, by training paradigm:

SGD pretraining with RLHF: GPT-4 -> turbo -> 4o

SGD pretraining w/ RL on verifiable tasks to improve reasoning ability: o1-preview/o1-mini -> o1/o3-mini/o3-mini-high (technically the same product with a higher reasoning token budget) -> o3/o4-mini (not yet released)

reasoning model with some sort of Monte Carlo Search algorithm on top of reasoning traces: o1-pro

Some sort of training pipeline that does well with sparser data, but doesn't incorporate reasoning (I'm positing here, training and architecture paradigms are not that clear for this generation): gpt-4.5, gpt-4.1 (likely fine-tuned on 4.5)

By performance: hard to tell! Depends on what your task is, just like with humans. There are plenty of benchmarks. Roughly, for me, the top 3 by task are:

Creative Writing: gpt-4.5 -> gpt-4o

Business Comms: o1-pro -> o1 -> o3-mini

Coding: o1-pro -> o3-mini (high) -> o1 -> o3-mini (low) -> o1-mini-preview

Shooting the shit: gpt-4o -> o1

It's not to dismiss that their marketing nomenclature is bad, just to point out that it's not that confusing for people that are actively working with these models have are a reasonable memory of the past two years.

While I appreciate the distinction you're pointing out, I disagree with your conclusion that the agentic system and its environment will remain separate going forward. There are strong incentives to merge the external environment more closely with the model's environment. I can imagine a future where GPUs have a direct network interface an os-like engine that allows them to interoperate with the external environment more directly.

It seems like a natural line of progress as RL is becoming mainstream for language models; if you can build the verifier into the GPU itself, you can drastically speed up training runs and decrease inference costs.

That’s categorically false, world GDP is not growing at 10%. If it was, we’d be doubling the economy every 7 years. That’s a staggering rate of growth

Then you should probably stop using most applications today, specifically those that have been iterated upon in the past two years.

Holy shit, I’ve been writing python for 15 years and it’s the first time I’ve seen None used as an identity function. That’s nuts! How does it work under the hood? Does filter have a special case for evaluating None?

Aaron Swartz, while an infuriating tragedy, is antithetical to OpenAI's claim to transformation; he literally published documents that were behind a licensed paywall.

> Inference costs, not training costs.

Why does training cost matter if you have a general intelligence that can do the task for you, that’s getting cheaper to run the task on?

> for quick mockups they’re serviceable

I know multiple startups that use LLMs as their core bread-and-butter intelligence platform instead of tuned but traditional NLP models

> take the word of domain experts

I guess? I wouldn’t call myself an expert by any means but I’ve been working on NLP problems for about 5 years. Most people I know in NLP-adjacent fields have converged around LLMs being good for most (but obviously not all) problems.

> kind of an insult

Depends on whether you think OP intended to offend, ig

Why is it relevant to include that factor

How many people can scrutinize the esoteric ensemble of climate models that existing climatologists use to generate weather predictions today?

What’s the definition of empathy? To me the connotation has always been, “is able to feel the feelings of others” as opposed to sympathy which is more about “imagining the feeling someone is experiencing”.

Regardless, aren’t we all trained in the same way - reinforcement of gestural and linguistic symbols that imply empathy, rather than being empathetic? I guess I’m wondering if hijacking our emotional understanding of interactions with LMs is that far off from the interaction manipulation that we’re all socialized to do from a young age

You’re right that some assets have been packaged to negative effect but there are clear examples of the opposite - futures are stabilize commodity prices, MBSs decrease volatility for investors which increases access to underserved SES profiles, carbon credits incentivize companies to decrease emissions, IP creates an incentive to innovate in a way that the public eventually benefits from.

I’d be curious to know what distinguishes assetization that creates negative externalities vs positive. Probably something about assetization that decouples the value of the underlying good or service from the created asset?

You can fuck a program up by calling an integer that you thought was a function but first class functions are still useful.

Thanks man. Basically sounds like they haven’t gotten better.

We have an advisor that’s pushing us towards langgraph but he’s the CAIO at a LargeCo and they have an entire team that’s devoted to building tooling for working with it. Which in and of itself is insane - the whole point of using a library is to avoid doing it yourself - but is also damning against LG because as an early stage startup I can’t afford the time sink of needing to work around a weird abstraction.

As someone who voted for Harris, I’d say trump is less ideologically driven than democratic candidates. He’s insane and erratic, which don’t follow ideology.

Isn’t there already a different incentive system in place? LLM providers are selling the search engine not selling ads on the search engine

Good on him, he’s getting ahead of bad PR before it can catch up to him and his company. That sounds like great leadership.

Yes, and when we're productive enough to do it, it'll probably happen.

What you’re describing is emergence, but it doesn’t mean that an emergent system doesn’t have mechanistic underpinnings. Your body gives rise to consciousness, but it doesn’t mean that you can’t treat blood pressure with medication.

function calling provides a "hint" in the form of a JSON schema for an LLM to follow. the models are trained to follow provided schemas. If you have really complicated or deeply nested models, they can become less stable at generating schema-conformant JSON.

Structured outputs apply a context-free grammar to the prediction generation so that, for each token generation, only tokens that generate a perfectly conformant JSON schema are considered.

The benefit of doing this is predictability, but there's a trade-off in prediction stability; apparently structured output can constrain the model to generate in a way that takes it off the "happy path" of how it assumes text should be generated.

Happy to link you to some papers I've skimmed on it if you're interested!

This feels like profit-driven marketing that boils down to “cheap old reliable solution bad, new expensive low entropy solution good”

it's totally wild that your first response is shitting on flaws rather than having your jaw drop at machines producing coherent videos from text.

This is _the worst that machines will ever be at this task_, and most of the improvements that need to be made are a matter of engineering ingenuity, which can be translated to research dollars.

I think the statistics he cited actually make that case