Hacker News Reader

Comment by falcor84 | Hacker News Reader

> Even large context windows will start failing

What do you mean by that?

If you have a large code base, a software engineer has to look at many files, and step through a big stack to figure out the bugs. Forget about concurrency and multi-threaded scenarios.

I’m assuming that an LLM will have to ingest that entire code base as part of the prompt to find the problem, refactor the code, add features that span edits across numerous files.

So, at some point, even the largest context window won’t be sufficient. So what do you do?

Perhaps a RAG of the entire codebase, I don’t know. Smarter people will have to figure it out.

Some LLMs already have a context window of 1M tokens, which I believe is already more than any human dev, but yes, I agree that it's not enough to look at it statically. Rather a multistep approach utilizing RAG and/or working directly with the language server would be the way to go. This recent post from Aider about using o1 as an architect seems to me like a good move in this direction - https://aider.chat/2024/09/26/architect.html

Hi. I built an AI coding tool that bypass this problem by allowing human to select the relevant code context instead of passing the entire codebase or using RAG, which has problems with precision and recall.

It takes a bit of effort but the result is much better.

You can check it out: https://prompt.16x.engineer/

That’s an interesting approach and I can see it working, at least working better than the default. It’ll at least work as a stop gap, till there’s a better option. Good luck!!

Also, some unsolicited advice if I may, your pricing is a little wonky, you may want to rethink that. I hate subscription pricing where it doesn’t make sense, but in this case subscription is a better option. Also, your team pricing should be per seat, perhaps with some tiers.

I like to know who is behind a tool, in your case that info requires several clicks. Also, use a more pleasant palette.

Edit: you should do a ShowHN..

Thanks for the suggestions.

The app is currently on lifetime license without the need of an account, so it is quite tricky to implement subscription without overhauling the entire app and workflow. I am going to raise price to match the expected LTV of a user multiplied by a discount factor.

For the landing page suggestions, I will see what I can do to put more personal touch, for now it is meant to look authoratative and professional.

I listened to the Cursor team on Lex Fridman yesterday. The biggest thing I took away is they have some wild ideas with having agents running in the background that are following what you are creating, trying to find bugs.

I understand what you mean but surely those guys were thinking about this awhile ago now. That part seems obvious listening to them. They are thinking way beyond that.

Reply on news.ycombinator.com