balder1991

I’ve been saying that. When you have no idea what you’re doing, you *need* the latest greatest model because it’s the only way to reduce errors.

For people who have some expertise, the models accelerate the grunt work, but you’re the one validating it.

Also the less you know, the more your solution will be the same as everyone else’s.

This is only true as long as they don’t think about the future or do risk management.

This must be why every Google app now freezes for 3 or 4 seconds on iOS when you launch them.

There’s a middle ground.

I’ve been using agents to do the code for me, but I explicitly tells it to iterate in small changes that I can review and approve as it happens, not after a 2k lines diff. It makes me understand what’s being made, and test every little change before we move on to change other parts of the code. Also helps you correct any misunderstanding right from the start.

I suppose the same way that the normal programmers and artists would call LLMs an attack on licenses and copyright.

That’s my biggest critique of this. I am working on a project with another dev and I try to keep docs as clean and “straight to the point” as possible. In my previous team, one dev went full “spec driven” and the project started having dozens and dozens of markdown files with hundreds of lines to the point any human being would give up trying to trim it. It just becomes noise, a lot of it not even correct because the specs themselves were mostly AI generated with lots of verbosity and wrong assumptions.

One thing doesn’t cancel the other?

Except there’s no average person, the average itself is a combination of people with the most diverse needs and constraints. It’s like the joke that the average human being has 1 breast and 1 testicle.

Well, at the very least the ability to have a larger browser screen on landscape or having two apps side by side is already good on its own.

It is a thing to some extent, but there’s also a correlation of companies that do cross-platform for simple apps being cheap on development, and making it work very well on some specific platform isn’t their priority.

In fact it’s more important that the right people find it, and it could be linked in a profile etc.

I think because the only places you can download it from are either from the App Store or logging into Apple’s developer’s website.

Basically, I’d say the LLM doesn’t have a “theory” of anything that wasn’t in its training. It’ll always treat your project as if it was reading a book for the first time.

As an example, if you chat with an LLM it can sort of explain concepts it learned in its training, because it created some internal representation of them. However it’ll never be able to explain things about your project unless you tell it (then it can simply repeat it, which is the same RAG does, the LLM is just spitting it out without any deep understanding of it).

You can try to cram as much context to it as possible and overload it with extra information, or you can understand this difference and separate a task that you know the LLM can handle because you’ll be able to leverage its knowledge and judge the final work yourself.

But this distinction needs to be more clear, I think. The LLM doesn’t have any deep understanding of anything you’re creaming into the context window the same way it does with concepts it learned during training from millions of examples and building some internal model of the concepts and their relationships.

Often times to write a good enough specification in a way the LLM won’t misunderstand it, you need to really understand the code and the possible pitfalls in it though. A few times when I tried this route, even adding details just in case the LLM still went and assumed something wrong that I didn’t consider would be a problem until the agent had to make that choice. Then it becomes a game of doing it again and again with more and more detailed instructions until you could have yourself done it in less time and having a deeper understanding of the problem (or I could argue having any understanding of the problem at all, since having the LLM do it for you is like being a manager who has no clue about the lower level decisions made and only being communicated the final result).

You’re not wrong, but there’s a nuance here. More often than not our thoughts fool us into thinking we have something clear when we have only a vague idea of what you want to do.

I kinda [wrote about this before](https://dielsonsales.github.io/2023/05/11/starting-a-new-blo...) but the takeaway is that writing forces you to structure your thoughts, giving them a final shape and enables you to notice flaws or gaps in your own thinking process. It’s impossible to do this without writing.

I’d like to recommend people to read the “Programming as a theory building” article and why the code itself isn’t enough to understand the whole context of a complex project.

Every time you work in a complex software project, you have all the context in your head about what you must take into consideration. LLMs don’t, you have to explain every little detail to them, but there’s no telling where it stops: do you have to explain X to it or is it in the training already?

When you try to shape how a LLM should behave and what it should know, you end up writing a Bible of relevant context that’s increasingly difficult to maintain as well, often with outdated or contradictory information, and it is still free to ignore things you have written and proceed however it wants.

That’s why so many people only feel comfortable giving the LLM some limited task to do, because you can judge if that specific task needs just enough context that it can handle by itself.

It’s what’s called in software engineering as “casual software” as a differentiator of “business software” and “critical software”. Not all types needs a high bar of quality, and most of the software engineering thought practices are tailored for business applications that will be made available to multiple users.

As you said, building a script that only you use personally or a very simple thing that just accomplishes one task and it’s easy to test require almost no engineering, and an LLM can often build those with very little downsides.

There’s even a framework already: https://start.solidjs.com/

Feels a bit like the AI browser that in the end imported Servo for all difficult things.

The way I see it, this is only possible because you can trust the lower layers work reliably and predictably enough that you can move up.

If your operating system was regenerated every day slightly differently and with certain things working and others not, you’d quickly revert to the lower predictable abstraction.

> it spends the 6 hours waiting for the tickets to go on sale and buys them for you

Or not, because other 7 billion agents were also waiting for it.

I guess the thing here (which they admit in the post) is that they’re just porting it to Vite, which is the real champ of the story. The LLM basically worked as a translator instead of rebuilding the whole thing from scratch.

So maybe the project is sort of maintainable, as long as people maintain Vite.

Actually, most VPN providers explicitly label the virtual locations as such, I think the famous ones at least do it (ex: Proton and NordVPN even explain them in their respective docs).

It depends on whether the VPN is lying to you. Proton, for example, makes them quite explicit in the software and even lists them for you here: https://protonvpn.com/support/how-smart-routing-works and seems like NordVPN also has a page explaining that.

Yeah, Proton is quite explicit about that: https://protonvpn.com/support/how-smart-routing-works

I have this impression that LLMs are so complicated and entangled (in comparison to previous machine learning models) that they’re just too difficult to tune all around.

What I mean is, it seems they try to tune them to a few certain things, that will make them worse on a thousand other things they’re not paying attention to.

Anything that is very specific has the same problem, because LLMs can’t have the same representation of all topics in the training. It doesn’t have to be too niche, just specific enough for it to start to fabricate it.

One of these days I had a doubt about something related to how pointers work in Swift and I tried discussing with ChatGPT (don’t remember exactly what, but it was purely intellectual curiosity). It gave me a lot of explanations that seemed correct, but being skeptical and started pushing it for ways to confirm what it was saying and eventually realized it was all bullshit.

This kind of thing makes me basically wary of using LLMs for anything that isn’t brainstorming, because anything that requires knowing information that isn’t easily/plentifully found online will likely be incorrect or have sprinkles of incorrect all over the explanations.

It doesn’t really solve it as a slight shift in the prompt can have totally unpredictable results anyway. And if your prompt is always exactly the same, you’d just cache it and bypass the LLM anyway.

What would really be useful is a very similar prompt should always give a very very similar result.

At least it used to be true.