jdauriemma

Better than 90% of the slop that gets ladled into the front page, bravo. The world needs more Clock Simulator-like projects.

It’s AI booster fanfic, but it’s well-written so kudos. The charts are slop though.

This team is at Stanford, unless I’m reading the article incorrectly. Still awful that the US pulled out of WHO.

That's the sad part. Empiricism is scarce when people and companies are incentivized to treat their AI practices as trade secrets. It's fundamentally distinct from prior software movements which were largely underwritten by open, accessible, and permissively-licensed technologies.

Can you update the title? "Do you have any evidence that agentic coding works?" sounds like you're seeking verifiable data, but in your post you're soliciting advice for how to use these tools. I clicked hoping for empiricism and got the same hamster wheel of "you're holding it wrong" that I see in every social media comment section about AI tooling.

Heads up: I clicked a tile and got the hamsterdance music pretty loud in my ears. Serves me right for being distracted during a meeting but still, sound warning :)

Example?

The big difference is accountability. An LLM has no mortality; it has no use for fear, no embodied concept of reputation, no persistent values. Everything is ephemera. But they are useful! More useful than humans in some scenarios! So there's that. But when I consider the purpose of conversation, utility is only one consideration among many.

The narrative structure of the article would be brilliant satire but I'm 90% certain that the author is serious about the conclusions they drew at the end, which I find sad.

Framing obvious/old ideas in a novel way can be helpful. "Collaboration" is a meme that makes it easy to view all collaboration-shaped exchanges as helpful and the opposite as harmful. A memorable counter-narrative can help people think more critically when faced with an applicable situation.

I don't think the author is saying that LLMs are conscious or alive.

I just wanted to thank the creators of this app for making it usable without having to create an account. It's a breath of fresh air!

> academic fraud

I would be content with just a statement like "these are real people I personally know (anonymized)."

Realistic fiction is still fiction!

I don't see any attributions or citations.

The analysis is interesting but depends on accepting a premise that is only substantiated by fictional anecdotes.

Final boss of z-index

I think I might have phrased my criticism too sharply, sorry about that. I wouldn’t have commented if I didn’t think you were making a worthy point though! There was an article about malleable software recently that used avocado slicers vs. kitchen knives as an analogy; the more I think about your piece, the more it resembles that concept.

https://www.inkandswitch.com/essay/malleable-software/

I feel like this is doable with a good LLM-assisted coding tool, but it's just a hunch.

Good point, though not all spreadsheets are created equal. Some get quite unmanageable, and that can be a productivity bottleneck over time (unless you're not really adding new use cases)

Excel is one of the most successful pieces of software of all time, so it's an odd choice for a punching bag. Plus, the tone of this article is (unintentionally, I'm sure) off-putting, particularly:

> You know what happens with Excel. Karen from accounting builds a “simple” spreadsheet to track expenses. Six months later, it’s a 47-sheet monster with circular references and VLOOKUP formulas that would make a mathematician weep. The company depends on it, nobody understands it...

It sounds like Karen did a valuable service to the company. She combined a technical skill set with her domain knowledge to create a system that was so successful that the company now depends on it. Cleaning up the technical debt seems like a task that's well worth the cost.

> ...And when it breaks (not if, when), guess who gets called to fix it?

I'm not sure, honestly. It depends on whether you want the fix being done by a resource you consider to be a cost center or a value center. The former will do the cleanup job for bottom dollar. The latter would team up with Karen to amplify the project's impact while cleaning it up.

For these reasons I think your analogy is not effective. I don't disagree with your thesis, though; I prefer code in almost all cases. But that's just me. I know for certain, though, that if I had stock options in that company in this hypothetical scenario I'd rather keep Karen around than whoever fixes the VLOOKUP syntax. And if visual tools are what empowers Karen... well, there you go.

This is similar to many experiences I've had with LLM tools as well; the more complex and/or multi-step the task, the less reliable they become. This is why I object to the "graduate-level" label that Sam Altman et al. use. It fundamentally misrepresents the skill pyramid that makes a researcher (or any knowledge worker) effective. If a researcher can't reliably manage a to-do list, they can't be left unsupervised with any critical tasks, despite the impressive amount of information they can bring to bear and the efficiency with which they can search the web.

That's fine, I get a lot of value out of AI tooling between ChatGPT, Cursor, Claude+MCP, and even Apple Intelligence. But I have yet to use an agent that has come close to the capabilities that AI optimists claim with any consistency.

How reliable are the results? I can expect a human with graduate-level execution to get this right almost 100% of the time and adapt to unforeseen extenuating circumstances.

That's great to hear - do you know what success rate it might have? I've used scheduled tasks in ChatGPT and they fail regularly enough to fall into the "toy" category for me. But if Operator is operating significantly above that threshold, that would be remarkable and I'd gladly eat my words.

> they're performing at at least graduate student level across most tasks

I strongly disagree with this characterization. I have yet to find an application that can reliably execute this prompt:

"Find 90 minutes on my calendar in the next four weeks and book a table at my favorite Thai restaurant for two, outside if available."

Forget "graduate-level work," that's stuff I actually want to engage with. What many people really need help with is just basic administrative assistance, and LLMs are way too unpredictable for those use cases.

The "in a box" phenomenon is very tangible to me when I am using the iOS Shortcuts feature. Its capabilities are so powerful, but its utility will always have a ceiling because app publishers' interests are generally not aligned with exposing a Shortcuts API to users. The more easily a user can automate and script the tasks that they use your app for, the less engagement their metrics will show.

Being accountable for telling the truth

I think LLM skeptics and cheerleaders all have a point but I lean toward skepticism. And that's because though LLMs are easy to pick up, they're impossible to "master" and are extremely finicky. The tech is really fun to tinker with and capable of producing some truly awesome results in the hands of a committed practitioner. That puts it in the category of specialized tools. Despite this fact, LLMs are hyped and valued by the markets like they are breakthrough consumer products. My own experience plus the vague/underwhelming adoption and revenue numbers reported by the major vendors tell me that something's not quite right in this area of the industry.

Okay thanks for your patience. I put together this gist based on an interaction we had with a user last year, I hope it's helpful: https://gist.github.com/bignimbus/457f9ac42f29d0bb6c4eb742ad...

That's a start! Forgive me, I'm heading into a meeting but I want to share some resources with you. It's close to the end of my day so I'll check back in the morning. If you prefer to continue the conversation on another platform, I'm happy to exchange emails or Discord handles. Also consider posting at community[.]apollographql[.]com, the Apollo Client team tries its best to respond to everyone who asks for help there. Any code snippets would be helpful to illustrate observed vs. ideal state when migrating.