“Chat GPT went down and 25% of my team could not work.”
twitter.com
twitter.com
Would love to hear real-world programming examples of something that would be faster done with ChatGPT vs Google/Github Search. What are they?
For instance I have a set of data that I want to make sense of before I can do something with it.
I can't remember from top of my head how to do things like calculate standard deviation or how to fit given array to a curve or how to visualise the data in this and that way.
I can get these code snippets from ChatGPT in seconds and they 90% of the time work.
That's just one of the example, but ChatGPT has reduced my time Googling for something by like 80%.
I also use it for things like constructing jq recipes, or zsh for loops (I still can't remember how to do those without looking them up).
Recent jq example: https://fedi.simonwillison.net/@simon/109882602515468465 - see also https://til.simonwillison.net/gpt3/jq
Here's an example where I used it as a search engine and it taught me about an entirely new (to me) tool: https://til.simonwillison.net/macos/sips
It saves so much time not to have to figure out how to instantiate that random call that wasn’t mentioned in the documentation and whose code is obscured by objectionable engineering.
However I don’t think any line of chatGPTs code has yet made it to my codebase unchanged
At first I was amazed that it could figure it out, until I started to scrutinize the code and realized it was entirely incorrect. It didn't produce a correct answer, just garbage that looks like a correct answer.
I assume ChatGPT will be better for more "broad" topics with more training data but in this scenario it totally fell on it's face.
Hard to keep using it after that.
asked ChatGPT how to export my tailscale config and it told me Whether you say ChatGPT is lying or say it's spewing bullshit or a hallucination, it's just plain wrong.
Tailscale is too new: ChatGPT's training is unlikely to know much if anything about it.
I wouldn't be surprised if Bing could handle this much better, since it has the ability to search for current documentation.
My most recent query from yesterday ways "iterate over a nodejs array, the array has 10 or less elements. The increment should be in chunks of two. Stop when the index is bigger than 10, or when there are no more elements"
I can figure this stuff out myself too, but with chatgpt, I can query it and come back in 20 seconds and use whatever the output is. 95% of the time is correct, and the rest I can just quickly fix it myself. Plus i can ask follow on questions and refine it even more. This stuff would have taken me 5+ mins looking over stack overflow and then coding it and testing it. Plus ChatGpt handles edge case so I don't have to worry about them.
The efficiency is much much better than SO. I can't imagine going back to Google or SO for any coding questions. I only wish they kept the service up more during the day time. It's available in the morning or at night, but most afternoons it's a hit or miss.
How many different languages can you do that in, in 20 seconds, without looking it up?
Personally, I think I can only do it in Python and C off the top of my head without looking. I'd have to look up how to do it, idiomatically, in C++, Rust, Go, Perl, PHP, and Javascript.
I'm not going to memorize random functions off the top of my head and I'm not going to waste brainspace on something I can use chatgpt really quickly and move on with others tasks more specific to my domain.
I'd rather focus and use energy on business logic + the part that are more specific to my domain than generic functions I can ask chatgpt and come back in a little bit.
It's like trying to bake a pizza and having to think how to grow tomatoes.
If ChatGPT can free you up from thinking about the basics, you can channel your energy into some more advanced concepts.
ChatGPT is literally 0 brain space for these simple tasks. If you had to code it you're at least thinking about edge cases etc. Plus there's 0% chance it would take you 20 seconds for 10+ tasks of these per day. if you're coding at 50 wmp these would take you at least 1 minute to type out, let alone implement correctly.
by your logic you should never use google or SO because it's quicker to just implement it instead of opening a browser, typing the question, looking at responses etc.
If I caught one of the other engineers on my team wasting time on google or SO looking up such basic stuff I’d question his career choice.
you seem to have an emotional reaction against chatgpt, or are just really dense. that's okay for you. but I hope your team knows that you're limiting their career growth by having you as a manager.
If you didn't code it you REALLY need to be thinking about edge cases, etc. ChatGPT (or stack overflow for that matter) makes tons of mistakes, at least as much as your average human.
Personally I find it harder to validate the behavior of code I didn't write than code I've gone through the creation process of myself.
Sometimes it is not perfect and a couple of times I had a case where ChatGPT couldn't code something and it would go into a loop coming back to the permutation of wrong answer over and over. Most of the time starting over with different prompt describing the same task in a different way fixes it.
Sometimes I feel it reads my mind. It's of course wrong many times, but the way it autocompletes code means, I am no longer hesitant writing a lot of code just to try things out, as it is much more enjoyable - this combined with ChatGPT really feels like having a superpower.
Similarly, I've saved hours or days on bug investigations by describing the bug and letting gpt guess why I couldn't reproduce it, and it was right.
I can certainly believe a significant productivity drop.
Specifically, I have a colleague I respect a lot. He is an excellent programmer. He now routinely asks ChatGPT for things and claims it's a big help. Yesterday I was wondering out loud what the difference was between two very similarly named properties in a config file, and he immediately logged on to ChatGPT and asked it. I read the answer and became suspicious because the explanation was what you'd expect the difference to be given the different names, but it didn't make any sense when you thought about it carefully. I typed a similar query into Google and the first result was a detailed and correct Stack Overflow answer. Now I'm quietly wondering about the other cases where he used ChatGPT to help him write code. When I tried ChatGPT to get library recommendations per tasks, it would often recommend libraries that sounded like they should exist but didn't.
Other people say they use Copilot and it's a huge boost. I tried it when it first came out. It kept suggesting huge blocks of code that were just nonsense, in a very irritating over-enthusiastic-girlfriend-meme sort of way. I found it was slower to try and work with Copilot constantly checking its answers than it was to just rely on the IDE autocomplete and type system (which are quite good in the language I'm using). Also I found that carefully reviewing its suggestions was somehow more tiring than just writing the code myself.
Finally, when I look at the bugs and things I've had to do lately, it's hard to imagine (current gen) AI helping. Everything is a bunch of small changes all over the codebase, often quite subtle and in response to complicated bug reports. Who are these programmers who are blasting out boilerplate all day where AI never makes mistakes?
However, I've gotten used to being able to ask "how do I X in Y" when I don't know the language or libraries well. The generated answers are often subtly wrong, but they can save me hours of digging. After all, even if I'd come up with an attempt myself, it would also require refinement and tests. So they can often shortcut discovery.
Some things I was able to do only because of the bot. It would have been uneconomical to learn the tech enough to come up with a solution myself.
One of the weirder things is how often an adjacent refinement will be suggested when I'd ask about a specific detail. It would take my example, and not only modify in the direction I want, but also to follow convention more closely. When it modifies an example to follow an old api version, that is sad, but overall I find it useful.
> "They can still work and the company does not rely on them to generate revenue. It is just from an IC tooling perspective in a short amount of time, they have become dependent on it. They can still go back to less productive ways but just a usage behavior observation"
> I think for some it’s already as critical as wifi.
Personally I would like to add that for me ChatGPT is as critical as running water and sewage.
If that were the case, I would turn off ChatGPT M-Th at 6 pm and every Friday at 4.