3,969 karma · joined June 14, 2013
I'm ready to give up on having it even review my code at this point. It's been so frustrating. It hallucinates bugs, especially in places where "best practices" are at odds with reality.
Recently it informed me of a bug where it suggested the line of code in question couldn't possibly do anything because on Linux the specific stdlib behaved in X ways, but it was obvious from the line of code that it was running on Windows which doesn't have this problem at all. Of course, it doesn't actually mention that this is an issue on Linux, just that there is a bug here. It vomits up a paragraph of $WORDS explaining why this was a high-priority bug that absolutely needed to be fixed because it was failing in subtle ways. Yet the line of code in question has been running in production, producing exactly the results it is expected to, for ~3 years.
And this is just one simple example, of the many dozens+ of times it has failed this task this year. In that same review run, the agent suggested 3 additional "bugs" or other issues that should be addressed that were all flatly wrong or subjective. I'm at a point of absolute exhaustion with this sort of shit. It's worse than a junior half of the time because of how strongly opinionated it is. And the solution to this sort of problem is an endless amount of configuration and customization that will be forgotten about by all of us over time, leading to who knows what sort of knock-on effects (especially as we migrate from one model to the next). We have a guy on our team who has ~17,000 words in his agent and instructions files, yet he sees nothing wrong with this. I guess he just really loves YAML and Markdown.
No. I resisted for a bit but have started using it at work. Mostly because I believe usage is now being monitored. I'm in a very high-scale engineering environment involving both greenfield and massive brownfield codebases and the experience is largely a net loss in productivity. For me and some others who I've spoken to in my org, opting in is a theater that we're required to engage in to keep employment and not a genuine evolution of our craft.
These tools struggle with context once you get deep into a codebase with many, many millions of lines of code and sprawling dependencies. Even for isolated Python scripts or smaller, supporting .NET apps, the time spent correcting subtle bugs or bullshit, or just verifying the bullshit, often exceeds the time it would take to have written it from scratch.
Regardless, what I've observed is that these tools do nothing for the actual bottlenecks of software engineering: requirements gathering (am I writing the right thing?) and verification (does it work without side effects?). Because LLMs are great at generating text, they're actively exacerbating these issues by flooding our process with plausible looking noise.
Well that makes two of us. Character seems to mean nothing today.
He wants to build the AI that makes people's lives better. Okay. Did the people ask? Do they have a say? It's all very easy for a billionaire to say when it's just him and a couple of people in his cohort in the driver's seat.
Beyond that I'd like to simply know why he thinks any of this is his responsibility. It seems much more obvious to me that he simply found himself in the right place at the right time and is trying to seize it all for himself as if it's his to take.
I get the sense that Bill does care about money, and so does Larry Summers, so Mom's advice probably wouldn't have done much there.
And before anyone suggests that I start tinkering around with the settings, I have adjusted it and the damned thing just resets itself constantly.