maherbeg

That's awesome

I'm surprised the OpenAI Codex team doesn't donate like 10B tokens or something to the Rust team for performance

These look great! I still have a Kindle Voyage from 2015 that's going strong, though I do wish it had the warmer LED lights. It's nice to have some devices like this and my Fujifilm XT-30 that don't make me feel the constant need for upgrading.

Congrats to Google on this! I wonder when the labs will start requiring commits in spend. It must be gnarly to do capacity planning if users swap between models every few weeks.

You might like Cormac McCarthy's Border Trilogy then! Easily one of my favorite series (first 2 books are better than the last one IMO)

Yeah, but for some reason OpenAI hasn't setup multiple accounts to have completely separate profiles yet.

I mean think about the magnitude of services and conflicting information there probably is from the us government. Just getting clarifying information on the trusted traveler program is a pain today, and that's just a single service. Imagine trying to figure out medicaid eligibility, social security disability, and any number of other things that would be helpful.

Painfully this will fall to the court decide and create the precedence here. If this is an official government app, then the onus probably is on the government to provide correct information? Who knows, it could go either way and the courts could say "you're on the hook".

I don't think we have a good answer for liability here, like for Tesla FSD, Waymo, or even the current frontier models.

It's a work oriented Muse if you're familiar with Muse. A single thread abstraction to manage your work and delegate to other threads.

OK, there's a lot of negative comments here, but this is a great idea at a high level. It really is hard to figure out where to do a thing and it's also very easy to get phished. If they can figure out how to help people get all the services they're eligible for that'll be an amazing improvement for the people.

Yeah, what things can you write to statically eliminate the things that are not acceptable. What would you have to change in your prompting process to get that outcome? What bad patterns is it copying from the code base that maybe you should spend tokens fixing?

I do agree that they're not great at program design by default and that's where we as engineers should spend our time. Data structures and data flow are king. But once you suss that out, they're pretty good at writing the resulting code.

This is also where I disagree with dhh about just using lower level languages. Good abstractions make for excellent program understanding and we should continue to build extremely good building blocks that make program design naturally solid.

True, but if you can think about tests as "how do we validate that the outcome I want has been solved", then you can orient your tests around that. I do wonder if BDD style integration tests will end up being the path we go down.

Frontier models today don't really write incorrect code at the micro level. They do miss edge cases at the high level though, and that's what we want to test, is the scenarios.

The cat's out of the bag already. We can't undo the idea of LLMs or coding agents. If training progress stopped today, we have years and years of harness improvements to extract more performance out of today's models.

We also have open weight models too, and ways to host those at home.

Most people don't look at the assembler output of their C++ code (I used to write win32 programs in asm!). Most people don't look at the opcode instructions or JIT output of their ruby / python code. We're starting to work at a higher level of abstraction using LLMs. It's ok to be sad about it, but just being angry about it isn't going to change that there's a new world out there with a new skill set that's needed for honing.

lol yeah, our review bot does a cost based analysis and pauses itself until you re-resume if it goes over a threshold.

There's different layers of understanding the system. I generally care about high level data flow, concurrency and performance (batching, holding transactions too long, back pressure etc.) rather than the mechanics of how the code actually does a thing. I still look to see what the final output looks like and ask my agent questions on how it fits in the larger system and evolve things if necessary, but agents are pretty good at writing code if the rest of the code base looks pretty decent.

Yeah the codex app can deterministically poll and watch for you too. Consider it like an event based trigger, where the event can be anything you can dream of (like webhooks!)

There's lots more you can do! Use the model to monitor your deployments after they get deployed. Have them fix and watch CI issues for you. Run adverserial review. Automatically watch metrics every day and highlight performance regressions. Start reviewing your previous sessions to find ways to statically reject different failure modes and have the agent have more success earlier on etc.

Another thing to think about is, what would it take for you to care less about the understanding. Better integration / e2e tests? Performance validation? visualizing program and data flows? Better refactoring of your modules?

come to the dark side and join us on the desktop apps. A lifetime of using a terminal has me surprised with how much I enjoy the codex desktop app.

Is this community adjusted EBITDA?

I imagine the first year we'll be at the Junior eng level, and then after a while make our way up to Staff Engineer. Then we'll have a bunch of staff engineers arguing and protecting their domains and then we'll need a new benchmark.

Ok, but can I bring GPT-6 in as an agent as a software engineer, tell it to talk to these people and have it start solving engineering problems and continue on for a full year career wise?

maybe call it EngEmployeeBench

I've seen https://www.lucebox.com/ as an interesting option

Those will use different wafers, so unless that memory is allocated for unified memory vs gpu HBM it won't make a difference.

This is pretty cool. I'll wait to watch some videos on it, but I haven't quite found a tool that is excellent for both single player and multi-player.

I'd like to have basically the Codex Desktop App / Mobile app for my personal use, connected to local LLMs. But I'd also like to be able to have a shared space, where my family and I can work together to do things like plan trips. This would then store artifacts like our itinerary, ideas, html mini sites for the trip for us to work on and use together. Agents sit alongside us helping us plan and organize. Buzz is decent for this use case, but hasn't fully nailed the mark.

Thank you for being explicit with the math!

So yes, at that speed for sure. But if the speed goes up? or the ability to batch at the same speed goes up? The economics start to shift. The gap is much closer, and you'd end up with a box you can still use or sell later.

Subscription pricing is still the best though!

I think this is a bit of a crazy statement. Everyone expects Apple to somehow build a category leading product every year. I'd expect something innovative every couple of years

* the iPhone * the iPad * apple watch * airpods * unified memory laptops and computers

Those are all products that either created a category or changed that industry.

If I can get Sol level capabilities on a $20k machine, then it is well worth it for my employer to buy me that machine for work as a workstation. When you start paying in tokens vs subscription costs due to enterprise agreements, you really start to see how much cash utilizing frontier models at the frontier costs (and I'm efficiently using luna and other models where possible!)

I'll say that Gastown sounded absolutely crazy, but the idea of having orchestrator threads to manage your work and keep tabs on it, having validators to validate the other work etc. were generally the right shape. I think GasTown probably could have been really successful if there was a pared down version with more obvious names rather than the fun names.

I'm going to be thinking about this blog post for a while though because if you squint and tear it apart, there are probably really good generalizable pieces in here to take home for future models.

Some of these feel like solved problems effectively, so having them in the standard library is nice (at the expense of keeping these forever for backwards compatibility once a new tech replaces it). I do think having a larger standard library for common things (like golang) is the way to go. If a dependency seems to basically be installed by default everywhere, maybe it should go in the standard library.

I think part of this is to enable the interface on the web and other devices where you might not have a terminal. But yeah, agreed, it's wayyyy too many tools.