Hacker News Reader

Comment by rideontime | Hacker News Reader

Skipped to a random timestamp to see how it performed, and at 15:35, the AI explains that "C# does not support array or list initialization using brackets directly," a cleanup which, seconds previously, Resharper had suggested to the user. The presenters were too busy gushing about what a useful educational tool the AI is to notice the error.

View on HN

This happens distressingly often in LLM demos. Significant errors being papered over because people are so enthralled by the "look Ma, no hands!" effect. Just got out of a work meeting where the exact same thing happened with an internal chatGPT integration.

To be fair, that's a C# 12 feature that came out three weeks ago

Then the fact that Resharper already supports it, while we have to make excuses for the AI Assistant, is very illustrative.

What does it illustrate? The time that it takes to train a model?

They're more concerned with selling flash and giving demos and selling something that doesn't work as expected vs taking the proper amount of time to train the models with up to date info?

Perhaps it illustrates they have people giving demos that don't actually understand or notice obvious discrepancies? This doesn't inspire confidence.

Yes, this feature's only been 'out' for 3 weeks, but obviously the resharper team knew about it ahead of time and had a definition of the feature to develop against. Are we saying the AI can only be trained on 'released' stuff, but other tooling can work against upcoming specs?

Do these models have any ability to learn? If it is wrong about something, can the user correct it or ask it to read some release notes and have it integrate the new information going forward?

Can't speak for this model, but I know from personal experience with GPT3.5-16k and GPT4, that if you get outdated code being generated, you can just include up-to-date documentation for relevant libraries and it gives you great current code. Alternatives examples include telling it that python now supports match case, and showing it the syntax expected for match case. It can then use this perfectly (though the current GPT line models know about match case, so no longer an issue)

I wouldn't call this learning, anymore than me knowing how to use a + screwdriver allows me to use a - screwdriver as well, but others might.

> The presenters were too busy gushing about what a useful educational tool the AI is to notice the error.

Having worked a bit in the education field, I think it's worrying that errors from AI applications aren't taken very seriously. I think errors in education is more serious than elsewhere because such error compounds easily: the person can keep a wrong thing in mind for years and spread it to others. A lie may even become a truth because very few individuals check sources.

We have the entire bullshit field of AI ethics already, it could start being half-useful if the problem of AI applications in education was taken under that umbrella.

Agree its a bit of a miss, but most of the battle with an LLM is pulling in the right context, so I'd imagine they could include resharper's output in the context in a future iteration. Seems like something they could iterate on.

Reply on news.ycombinator.com