Hacker News Reader

Comment by osigurdson | Hacker News Reader

I have a paid subscription to Windsurf using Claude. I struggle to find much utility in terms of actually writing code. It spends a lot of time trying / retrying and glitching out. I'm sure in a couple of years it will be amazing (assuming a fundamental breakthrough isn't required) but it isn't quite there yet.

LLMs are super useful but currently, the primary use case is teaching, not doing. For this reason, I think ChatGPT is really just as good as an AI enabled editor (or both if you don't mind paying for two subscriptions).

View on HN

To correctly vibe code something useful, I find that I need to religiously give faith to the tool and forget all about SWE principles and best practices and instead treat it like a child who makes mistakes and corrections while the adult must not intervene or admonish too much but rather nudge it to right direction.

Also vibe code has a parallel feature, while the code is generating, you are also doing live review and correcting it towards right direction, so depending on your experience, the end product can be a bad mess or wonderful piece of creation and maintenance dream.

The issue with seasoned SWE is that, the moment a mistake(or bad pattern) is made, the baby is thrown with bath water.

For a tiered app like the one presented, 35k LOC is not really that impressive if you think about it. A generic react based front end will easily need a large number of LOC due to modular principle of components, various amounts of hooks and tests(nearly makes us 25-40% of LOC). A business layer will also have many layers of abstractions and numerous impl. to move data between layers.

The vibe code shines, when you let it build one block at a time, limit the scope well and focus. Also, 2-3 weeks is a lot of time to write 35k LOC. at start of any new project, LOC generation rate is very high. But in maintenance phase it significantly falls as smaller changes are more common.

I've had a bit more success on the front end as it is possible to see the results of a change very quickly. In fact I would prefer it to just auto-apply the changes and I would visually inspect. It isn't bad but the workflow is pretty slow. The resultant code is also very verbose - likely 3X more code than an experienced engineer would create (this is one part that I am certain will improve dramatically in time). While I do use this workflow, sometimes I feel like I am just being lazy as opposed to productive.

I'm just being honest. For my use case, I would be much better off if LLMs could just do everything.

My experience matches yours.

Lots of apps are quite repetitive: for building APIs for example you generate one controller and the ask the app to generate more using the first ones as a pattern. For frontend you do the same for forms or lists.

Tests are often quite good, but I think they were already great even back in the first ChatGPT release.

With this strategy and the fact that some patterns are quite verbose (albeit understandable for an AI or a reader), it is quite easy to get to a big LoC while still maintaining consistency.

> LLMs are super useful but currently, the primary use case is teaching, not doing.

For code? Autocomplete on steroids is the killer-app.

The other things the LLMs give me are prone to be over-engineered/overly verbose code or similar.

I went through a lot of "Why are you also doing $FOO then $BAR? Doesn't seem necessary if we skip them and do $BAZ which will make one or both of those redundant" and it responding "You're right! Lets use $BAZ instead".

And giving them code to make a small change to was pointless - they would often, but not always, make an incidental change far from the point where you asked for the change.

But autocomplete? That works just great and because I've already got context of the code I am writing I can check it in (at most) two seconds and move on.

I think you're missing the value of rubber-ducking with the chatbot on a fairly well-defined/well-known problem in a somewhat specific/novel scenario—this is the vast majority of software development being done today.

Depending on the situation this can be invaluable. If you're experienced in the domain you probably know generally what you need to do but you might get a better result by reasoning through the best solution with the constraints and requirements you have. Or maybe you missed something obvious when you write out the full context—which is a required step for getting a good output from the chatbot, and generally isn't a required step if you aren't explaining your approach to someone else and you don't want to be rigorous.

> I think you're missing the value of rubber-ducking with the chatbot on a fairly well-defined/well-known problem in a somewhat specific/novel scenario—this is the vast majority of software development being done today.

I actually do use ChatGPT for rubber-ducking, but in that context there is no (or very little) code. In a coding context, I've resigned myself to purely autocomplete-on-steroids.

The thing is, in the vibe-coding context (having the LLM write the code for you), I've had atrocious results across all of the popular LLMs.

This does seem like something that is worth getting to the bottom of. Why are some people having such good results while others do not. What is the secret sauce?

I have very similar experience.

After seeing how people like Andrej Karparthy used vibe coding to generate applications https://x.com/karpathy/status/1903671737780498883?s=61 I realize that

you need to be clear on what you want the LLM to do break down the tasks and give byte sized tasks to llm to do specific thing and sometimes I had to tell it not go and change random files because it found the need to refactor them.

Full Disclosure: I'm building a LLM chat app for software developers and domain experts

> I struggle to find much utility in terms of actually writing code.

I personally feel you need to give up some control and just let the LLM do its thing if you want to use it to help you build. It honestly does a lot of things in a more verbose way and I've come to the conclusion that it is an LLM writing code for another LLM. As long as I can debug it, I'm okay with the code, as I can develop at a pace that is truly unreal.

I finished my "Recent" contexts feature in a half a day, today. Without the LLM, this would have taken me a week I think. I would say 98% of my code in the past few months has been AI generated. You can see a real life work flow here:

https://app.gitsense.com/?chat=eece40e2-6064-46d2-9bf1-d868c...

I truly believe if you provide a LLM with the right context, it can meet your functional specs 90% of the time. Note the emphasis on functional and not necessary style. And if *YOU* architecture your code properly, it should be 100% maintainable.

I do want to make it clear that what I am doing right now is not novel, but I believe most problems are not. If the problem is not well understood, it can be a challenge like my my chat bridge feature. This feature allows you import Git repos for chatting but I will probably need to rewrite 50% of the LLM code since the solution it built is not scalable.

I recently asked Claude 3.7 (Thinking) to help me debug. I told it that it seemed my code crashed when MY_FLAG was true. Its solution? if MY_FLAG: MY_FLAG = False

Do you come across issues like this too or am I not prompting it correctly?

Reply on news.ycombinator.com