Hacker News Reader

Comment by IshKebab | Hacker News Reader

Yeah enjoy it while it lasts. This is basically where LLMs were with coding a year or two ago. Give it a couple more years...

View on HN

I just started using opus 4.8 and it's still trash at very domain specific stuff and debugging. It went down a rabbit hole trying to coerce the algorithm to work for 1 data point, which dramatically overfit the code. Later it did 11 experiments to try and figure out how to do the math, each one yielding an accuracy no better than a coin flip and each one it asserted beforehand that it had it this time, 100% accuracy.

We've got some time.

That's last gen though. I agree it's mediocre. Try Astra. It's significantly better than Opus and even Fable. I'm not sure about earlier OpenAI models.

I also thought we had time.

They say that every generation. https://www.youtube.com/watch?v=akNcZOoLKVk

I didn't say it was AGI, I just said it wasn't trash and that it's significantly better than it used to be.

Anyway just because AI sales people continually jump the gun doesn't mean AGI is never going to happen. It just means you just have to try the models yourself to see how good they really are, which I have done. Astra is really freaking good. (If you aren't paying.)

I suggest you give it a try and base your opinion on that.

It seems to depend on the domain, and perhaps the tech stack? I’ve been disappointed by Astra, it’s worse than both Fable and Opus for me.

Yeah maybe. I dunno though, I'm using it in obscure SystemVerilog stuff. AI used to absolutely suck at that very recently but now it seems to have no problem.

I think the key is that you give it all of the info - put all the docs, the LRM, tool manuals, etc. in a directory so it can read it if it wants (it really really wants).

Reply on news.ycombinator.com