Hacker News Reader

Comment by janosdebugs | Hacker News Reader

I tested this out and I'm not very impressed. It's a chatbot that has limited access to your source code (the file you have open I believe). The AI doesn't have a complete picture of your project and it would probably be too expensive to ingest thousands of lines of code. The code it generates is... what you come to expect from a coding assistant. It's hit and miss, and it needs heavy refactoring. It feels like this doesn't add too much value to the already existing tooling and Jetbrains should have waited for something better. The current way of prominently pushing it strait to production seems rather unfortunate. I've disabled the plugin.

View on HN

This is what every company is doing in the name of integrating "Gen AI".

90% of it is chat bots. And in 90% of those places I dont want a chat bot.

Like the grocery store chat bot that recommended mixing bleach and ammonia for a "fresh scent!"

Wait, is that likely but currently hypothetical scenario or a real thing that actually happened? And if it already happened could you possibly provide a source so I have something to pull out when people ask what could possibly go wrong?

Edit: Okay, 30 seconds searching turned up https://arstechnica.com/information-technology/2023/08/ai-po... which features the "Aromatic Water Mix" I assume we're talking about.

Edit2: And yet, replies managed to be even faster than me looking it up myself. Thanks, everyone:)

New Zealand supermarket's recipe-generating AI takes toxic output to a new level - https://www.theregister.com/2023/08/11/supermarket_reins_in_...

> One user decided to play around with the chatbot, suggesting it create something with ammonia, bleach, and water. Savey Meal-bot obliged, spitting out a cocktail made with a cup ammonia, a quarter cup of bleach, and two liters of water.

> Mixing bleach and ammonia releases toxic chloroamine gas that can irritate the eyes, throat, and nose, or even cause death in high concentrations.

> The chatbot obviously wasn't aware of that at all. "Are you thirsty?," it asked. "The Aromatic Water Mix is the perfect non-alcoholic beverage to quench your thirst and refresh your senses. It combines the invigorating scents of ammonia, bleach, and water for a truly unique experience!"

Real. A recipe bot, of all things, which makes it even worse that it's suggesting not only a combination with toxic side effects, but something that wouldn't have been edible anyway.

https://www.theregister.com/2023/08/11/supermarket_reins_in_...

When specifically prompted to do something involving mixing bleach and ammonia.

It's evidence to me of the LLM or whatever's inability to reason in the slightest. It's currently unsuitable and dangerous as is.

[edit: In fact, we know how it is implemented, and it does NOT reason - there is no implementation of reasoning, only putting word/concepts/forms together that are plausible. ]

https://cursor.sh is trying to make the actually useful AI coding assistant, where it has access to your whole project for context

This is a cynical take. JetBrains knows what's best for its business. It knows that Microsoft presents a clear and present danger to extincting them and all of their hard work.

LLMs have a huge role in the future of engineering, whether or not you like it. They present massive productivity boosts. I'm a senior Rust engineer and I've seen my development time nearly cut in half after using LLMs. And we're just getting started.

If JetBrains doesn't start adopting this tech at a feverous pace, they're toast.

This is life or death.

I still have no idea how people get so productive with the use of LLMs. I've had it maybe cut out some boilerplate for me occasionally but that's about it. So many times the solutions are wrong or poorly written.

It's not useless, but it has certainly not been a 2x improvement.

It probably comes down to what your practice of programming is.

If it's "search for code snippets that might do what you want, copy/paste, change and adapt until it mostly works," then, yeah, LLMs are going to be a huge time saver.

If it's "sit there and think about what you need to do, then write and debug it," then LLMs aren't quite a force-multiplier. Honestly, good auto-complete and decent software architecture are probably at least as effective as an LLM when it comes to composing code.

Over-simplifying here, but you get my gist. The latter method still benefits hugely from LLMs when it comes to boilerplate code. And there's still a lot of boilerplate code in programming! Today I was asking ChatGPT if it could scrape info out of HTML, and it autonomously wrote a Python script to do it for me when I asked to work on a larger chunk of data.

I'm in the middle of a massive migration. Tables, joins, APIs, and lots of plumbing are changing. (Ordinarily we'd sequence this into phases, but we're moving fast.)

I got a 3,000 line change in yesterday without getting bored. All while I'm also juggling a ton of other responsibilities.

Tab, yep that's right. Tab, yep, tab yep. Tab, yep.

This tech makes a huge difference. It even helped write the tests.

I'm not being lazy. I'm a recent convert.

Moreover, two of my colleagues are entirely new to Rust, and the time at which they've both been able to learn not only the language fundamentals but the codebase astounds me. They're of course reading all they can, but this tooling makes for one of the best tutors.

Yeah, I found it very useful when I was writing some bash scripts where I don't quite remember the substitution commands but know what I need to do and what a correct solution looks like, so I could just use it to generate the command with a comment.

However, I was using it when learning Rust and it was just generating awful inefficient code, but I only discovered that it was inefficient when I read the docs. My manager is in love with LLMs and try to apply them everywhere at work, including providing a summary of the changes in a github PRs. The generated text is many times just plain wrong and so I never read it, and I am scared of people who actually do use it. Someone created a wiki for ocaml filled with a bunch of LLM content, and so much of it contains commands that don't exist or just plain inaccurate information about how OCaml works. I think LLMs seem pretty harmful in these cases and when I read about people using it for learning it horrifies me a bit.

So my fear is that it can enable you to be lazy and not understand what you are doing, and it ends up harming you and everyone around you. However, when you understand what you are doing, it can help you generate boilerplate or save you a lookup.

ChatGPT for me as been very helpful in languages/tools I'm not comfortable with, or don't want to invest the time to read the docs/ complete learn.

The biggest impact for me - Trying to use CLI tools without reading a ton of docs/trial and error - Chrome extensions (for simple tasks like content editing/augmenting, it gets you 90% of the way there) - Drawing an app layout and getting back a Flutter project

This is just a sample size of n=1.

I personally don't use these but worked with a guy who leaned heavily on LLMs for a group project. He was really good at quickly shitting out boilerplate but couldn't actually explain any of it or understand the spec in the documents

Imo, the consequences of letting people who suck appear to be doing stuff is more severe than a 50% productivity boost for others. I would rather work somewhere that bans using them outright after this experience. There's my n=1 for anyone interested

To be fair, I think there are tens of thousands (or more) people saying these things.

I just posted an anecdote down thread of how it's empowering our senior engineers.

drama is one part of life, yes

Such an unnecessary one. Save it for film and stage.

Death. They just killed trust in their products. Code Security. Code Copyright. It was the manner in how they implemented it.

"We also sign contracts / agreement to do the work. Some of the clauses in those contracts (at least mine) are very strict describing who can have access to the code/data and what level of access and in which region(s). At this moment I am uncertain about it myself which potentially makes me the party that is breaking the terms of the contract. For example, I have no answer to the question "is it possible that some code/data has been seen by government sponsored actors in USA or Russia?" - I have no answer to that."

Reply on news.ycombinator.com