Hacker News Reader

Comment by _jab | Hacker News Reader

I understand the sentiment, and I definitely think there are appropriate comparisons to be made to past hype waves like crypto and even AVs, but the fact that so many normal people use ChatGPT and find it not only interesting but also, critically, useful, tells me that LLMs may follow a different path in the long term.

View on HN

I think there are broadly two types of people, one who understand LLMs and realise that it is actually going to change the world, others who think it is just a hype like crypto / NFTs/ meta verse.

I have friends working in big tech who are completely unaware of how to use GPT - maybe because their work prohibits it and can't appreciate the value.

There's a third person--someone who knows what these tools are and knows their limitations mean they will only be applied successfully to very narrowly scoped problems and domains. I've been in and around the ML space for a couple decades and the hype for LLMs is just plain unhinged and does not match the reality of their applications. There are very few domains that benefit from a better BS generator, which is what these tools are in practice.

Im in this camp, my impression is it’s a tool that augments me while working, but it’s something that’s incredibly difficult to reliably use in a service: the sandboxing between user data and instructions is a real problem in langchain/react.

> so many normal people use ChatGPT and find it not only interesting but also, critically, useful

Is this actually true though. I write code for living and tried to use it many times but it wasn't really that useful to me ( vs google search)

Curious to hear ppl who are using it in 'critically useful way' . i am eager to use it my workflows .

To your point - I don’t actually know a single ‘normal’ person IRL who uses ChatGPT or any other LLM app in general. The only people I know that have ever even mentioned them are other engineers at my job. Even then it’s more just bragging in passing about getting into the latest beta $whatever or making it generate some outrageous paragraph for 30 seconds of fun.

I have seen several people justifying their use for it to work on that novel they always wanted to write. when challenged on who is actually writing it, they respond with "you need to understand the limitations", and that by itself it "can't write the novel". in short, they mean that ChatGPT only grnerates short passages the size of a few paragraphs. they are therefore just puzzling and joining these short plots together. can't imagine what kind of novel could be resulting from this.

We are already seeing stories of AI fail too, like this lawyer that tried to use it to write an argument and it invented entire cases which don't exist. The judge is not pleased! https://www.bbc.com/news/world-us-canada-65735769

Anecdote, but I needed some code to transform a couple of csv’s into one with some cells merged. I could have written the Python myself in about 45 minutes including the testing and spot checking.

Instead I described the csv’s that I had, the one I wanted, and asked for Python code to do it.

The code ChatGPT (v3) produced was flawless. I ran it on the data and spot checked it. The entire process took five minutes.

I was on vacation at the beach so this was back in march and I had just received access to gpt4. I had lot of time and I was mostly curious to see if I could use c++ and the recently released whisper.cpp which is pretty solid for transcription s… so I haven’t written c++ for sometime and figured let’s guide gpt 4 to help me write this. It started out giving me the Skelton code. From there it was less about it driving the code and more about it writing the tedious functions. A request handler for parsing form data. After I got the example I could reuse and fit the code into the right places at the end I have a working c++ service wrapping whisper.cpp … I should open source the work but I was blown away that I was able to get this working and still be totally relaxed… occasionally I’d compare my searches in google to my answers from gpt 4 and the difference was me spending 15 - 20 minutes reading through search results vs 5 to 10 minutes… so yeah I’m a crazy ceo who won’t shut up about ai

See this thread: https://news.ycombinator.com/item?id=36152510

It's not going to be barely useful, if at all, when you have high skill in a domain. It's quite useful when you don't.

It’s pretty useful for doing grunt work in domains that you know pretty well.

https://chat.openai.com/share/aff14574-4f3e-496e-a11c-aa8ee2...

I could have done the work by hand, but instead I played some guitar while it updated the labeler software for the ML project I’ve been working on.

In fact, I think ChatGPT and Copilot wrote the majority of the code in this project:

https://github.com/williamcotton/chordviz

I'm a bit confused by the productivity gain you are seeing. For me, it would be faster to just do the react work (assuming I'm in webstorm with its own fast autocomplete) than go back and forth into ChatGPT and explain what I want in natural language and copy the results back in

The only clear win I see is your last example where you are extracting coords into an array. I agree that text transformation like that is a great use of ChatGPT (that's mostly where I use GPT3)

Aside from copying and pasting, all I typed was:

Add a text input and button to this app that sets the currentImage cookie to the filename in the input and then sets the current image to that as well

And:

Great, now I'd like to add support for the ii and vi chords. We need to add the keyboard shortcuts for "2" and "6" so when these are pressed it correctly sets the chord name as well as updating the tablature

And:

The G and C are finished. Complete D, E and A

The cognitive load was significantly reduced to the point where I got through practicing Paul Simon’s America twice while waiting for the responses. And that’s not the easiest song to sing and play on acoustic guitar!

So instead of just having the updated labeling software, I got some practice in and retained some will power to label 1,500 images when it was completed!

But thanks for explaining to me that you know better than me about what saves me time and energy…

I have done some iOS and macOS work in the past using ObjectiveC and C. I have done some DSP work using MaxMSP and PureData.

This morning I decided I wanted to write a simple audio visualizer for my iPhone using SwiftUI, a language and framework I have never used. Within an hour I had through the direction of ChatGPT-4 an app running on my phone using a 3rd party library called AudioKit and drawing some fractal looking thingies, as well as a pretty decent understanding of how it all worked because I asked for detailed explanations of every line of code.

I’m not sure that I could have had this little app up and working in a single day let alone an hour if I only had a search engine at my disposal.

Googling "audio visualizer SwiftUI" gives me these as the first, fourth, and seventh results:

https://medium.com/swlh/swiftui-create-a-sound-visualizer-ca... https://audiokitpro.com/audiovisualizertutorial/ https://developer.apple.com/documentation/accelerate/visuali...

I do tend to reach for GPT4 before Google for things like this now, but I feel like it should definitely be possible to get this up in only slightly longer with just Google, even if you want some mods.

But I wasn’t interested in someone else’s audio visualizer, I was interested in a specific approach using pitch detection and primitive line drawings, not displaying an FFT as a bar graph. Sure, I could have synthesized a few different search engine responses but it would have taken a lot more work and I would have run into issues that slowed me down to the point where I had to get back to my other duties as the father of 2 and 4 year olds who were home for the week!

Yeah its definitely better than piecing together parts from different tutorials to be sure.

For sure. The argument that there was hype about something in the past, therefore any hype is unwarranted is not a logical conclusion.

Not only is it unwarranted it is more obnoxious than some YouTuber making predictions about all lawyers being replaced in the next three years.

I've yet to see a serious study where there are measured improvements to QoL vis a vis workload, time required, etc. vs accuracy and fallibility issues that have been comprehensively demonstrated with the current crop of LLMs.

That’s how folks initially thought of cars too - the accident rates, lack of speed etc. The point is, as a technology, this is going to change the future. It’s a valid point that the current state might be less than what you expect. The only way to improve it is to double down on it.

Comparisons to crypto are a bit shallow - except that they both are ‘new’ technologies and need GPUs, there’s not much.

One was just posted: https://news.ycombinator.com/item?id=36152510

I already have a massive improvement in productivity from GitHub Copilot that I don’t find it hard to imagine that fine tuned LLMs will be able to automate the bulk of mundane work in software (maybe even all of it). Programming, and other computer based activities that involve writing are the exact types of things that language models are good at.

You are probably right that in the six months since the release of ChatGPT, we do not have academic surveys of the productivity gains of LLMs in real-world scenarios.

But that is hardly the measure of a new technology.

Luckily, I was just reading such a study a few days ago: https://www.itnews.com.au/news/westpac-sees-46-percent-produ...

Reply on news.ycombinator.com