So enlighten us with some examples of useful things you found.
-Had it extract some gnarly string manipulation logic in a MySQL where clause into a function. It wrote clean, well commented code, and even explained what a regex did in a comment.
-Had it explain to me how macOS plist files worked, how to view them, how to edit them
-Asked it what some lines in an ssh config file did and it clearly explained it to me
-Dropped in some XML from a spring applicationcontext file that was doing something I didn't quite understand and it explained it to me. Asked how I would change something about it and it told me how, explained how it worked, and provided example xml
-Gave it a SQL Server CREATE TABLE statement and asked it to create the equivalent CREATE TABLE statement for MySQL 8, which it successfully did
My only complaint has been how sometimes the system has been unavailable. I now pay for the $20/month subscription and that has been resolved.
I asked it to write some queries in relational algebra and it was close but it was invalid in ways that a non-database person wouldn't understand. Kind of looked right.
Are you claiming that for all possible problems posed, verifying the correctness of the output of an LLM is equal or higher difficulty than solving the problem itself? If so, that seems like a claim more arising from emotion than reason.
In the general case, you can have it generate code in a proof based language that "proves" the code is correct against formal specification. Unless you consider math itself to be "bullshit" too.
They also probably never asked people to work on a task either (hint: people can get things wildly wrong, even generally competent ones)
As for proof based languages, can it actually do that? Have you tried?
It's not really any worse than asking someone knowledgable about a subject face-to-face. They know a lot, but they can be mistaken about things. At some point you will tease out the misunderstanding.
The best way to understand the value of the tool is to try it for yourself.
I can see how it might help with source to source translation or with generating random stuff, but that is not a problem I have.
Edit: on second thought, I believe "Editing ChatGPT output" might be a valuable job skill in the future, and maybe a good cover letter would give recruiters some signals in that way.
Not sure what to do about this. I want ChatGPT as a tool available to me, but I don't want any service provider I interact with to use it because I know they'll fuck it up eventually and I'll get worse service.
This is just straightforward selfishness and narcissism.
No software developer should be so naive to assume they'll always do the right thing with a powerful but incomprehensibly dangerous tool.
- Explained why a python variable in a closure was saying it was not assigned, after several minutes of googling/SO failed me (needed the nonlocal keyword, different from js)
- Explained the difference in several slightly-different C++ lists, which I couldn't google because the differences were mostly special chars
- Wrote working powershell code to output a list that has every string in $list1 that is not in $list2, on the first try, after I struggled for over an hour to do the same (and google/SO failed me).
- Explained the difference between two batfiles for the Intel Fortran compiler after google failed me
- Generated some really cool D&D statblocks
-Unbeknownst to us players, generated a really creative heist dungeon that our DM ran us through (he thanked ChatGPT as his "co-DM" at the end). This one is particularly notable because the man works at a hiring agency, he's not exactly a techie.
This is what I've been noticing as well. There have been many times when I spend 5-10 minutes trying to figure something out with Stack Overflow and Google, not getting anywhere, and then ChatGPT immediately gives me the answer. I'm not sure how anyone who considers Stack Overflow useful would think that ChatGPT is useless.
Do I need to verify the answers? Sure. But I have to do the same thing with answers on Stack Overflow. Even if I'm using official documentation, I need to verify that what I think it's saying is what it's actually saying (particularly when it's as unclear as it often is).
What's even more interesting is that I keep running into people who are finding completely different uses for it.
The critique here is like a lot of other critiques I've seen. They seem to come from people who haven't spent much if any time working with it, but have read some articles about how it's not infallible and decided that meant it was junk (though this piece goes the extra mile when it hints that big GPU is behind it all). What's striking to me is the lack of intellectual curiosity on display.
Tangent:
Google is less and less useful (as is SO for 'old' languages). I look into a mix of reddit, github issues, github code and SO depending on the issue. Hopefully this advice helps you if Chatgpt/copilot can't.
Of course, we shouldn't rely on chatGPT. It has give me wrong and insecure code before. However, its a nice tool to have around
You 100% should verify any code generated by ChatGPT - but this goes for any code found off the internet. I have come across bad Stack Overflow code committed to codebases before.