Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.
Every second that I didn't have to:
- go look up a similar problem on Google
- open three or four stack overflow tabs
- read through the stack overflow links
- copy out the answer
- change all the variables to match variable names that I want
All of this represents a huge time saver for me. I'm honestly baffled that people lack the ability to use LLMs in a productive and optimal manner.
in surprised how few services developers I’ve encountered who don’t know hot to configure a web sever or LB.
ChatGPT will only help you with the simplest of tasks, and even then you're gonna fail if you don't know to correct it. If you go beyond the most basic of the tasks you can learn how to do in a few hours, good luck fitting them in a chat prompt.
The amount of Kubernetes I needed was probably more than what ChatGPT can solve, but at the same time, I spent an awful amount of time looking at various templating solutions.
Docker is being replaced with Podman. Supposedly a drop-in replacement, but the tools built for Docker don't play nicely. You have Ansible, but there's also Puppet, Salt, Chef. There's Terraform and also CDKTF (from the same company too), Pulumi, plus tools from each cloud provider. There's Prometheus and Nagios, Datadog, Sentry, New Relic.
The irony of CI/CD tool of your choice is not lost on me.
I'll give you Kubernetes itself as the system.
Yes, the concepts (containers, orchestration, reproducible deployment, infrastructure as code, unified monitoring, continuous integration) are well-established. How you get there? Not at all.
Yes, the OP was hyperbolic when claimed the tools change every year. But the landscape of devop tools is changing. There are dozens of dev tool companies funded in the 2023 winter batch of Y Combinator alone. Someone sees potential to disrupt the established tools. And it will be disrupted.
There was a world before Kubernetes and Terraform. It wasn't that long ago.
https://www.ycombinator.com/companies?batch=W23&tags=Develop...
> Saying that DevOps tooling changes every year is absolute nonsense.
And then you list 5 technologies with an alleged lifespan of half a decade, which sort of hints at the idea that someone would need to learn/update a new tooling skillset once a year.
In that light, GP's comment doesn't sound so wild.
Huge +1.
I’ve got it to do a large part of my work for me by chaining some simple API calls.
The fundamental conceptual gap I see - people often ask it to do some “thinking”. Then are annoyed by the inaccurate output.
A simple example is doing word count and it getting the wrong answer very confidently.
Of course it sucks at that. It doesn’t have a counter internally. But if you ask it to number each word in the input and output a list, then ask for the word count, it gets it right every time.
Almost like how a human might count words manually.
Here is what I asked it to do the other day, and it just did it right away and got it entirely right.
> Write async tokio Rust code that takes in a Vec<u8>, writes it to a temporary file, calls /usr/bin/svc infer pathToTemporaryFile, reads and deletes temporaryFile+".out" and returns Result<Vec<u8>> containing the contents of temporaryFile+".out"
I knew what I wanted, I could have written it myself, there was no unknown but it took fewer keystrokes and honestly when asked to write small components like this with pedantic detail, it does an incredible job and the output is easy to validate quickly.
As someone who is far from an expert in DevOps, this has saved me a lot of time recently, and it worked fairly well to get past duct taping infrastructure together to then leverage.
In my experience, GPT saves me some time when I don't really have to think about idiosyncrasies anyway. In cases where I do have to think about them, GPT doesn't offload that from me. It often tries to mash unrelated frameworks together or makes up non-existing APIs that it would find useful for the given task. When it does work well, it mostly boils down to automated series of copy'n'pastes. Saves a bit of time, sure, but doesn't really transform the way I work.
- Article: https://simonwillison.net/2023/Apr/15/sqlite-history/
- ChatGPT transcript: https://gist.github.com/simonw/1aa4050f3f7d92b048ae414a40cdd...
If you just need the beginner's gist or are completely unfamiliar with something, they're not bad. Getting specific answers to corner cases is often a waste of time.
You always have to walk back and do more research to find the best practice.
And it doesn't learn. I stopped counting the times I told it it was wrong only for it to acknowledge the error and provide the same answer as a fix.
ChatGPT is nice to learn about new ways of doing things, new ways of composing things, but then a human must intervene to make sense of the mess that was generated.
If you want to make accurate predictions, you'll need reasoning or at least some process that approximates it.
"There a no bananas. Max is looking for bananas. Will he be successful: [yes/no]"
I just tried using GPT-4 to translate a random htaccess file I found to nginx and it seemed to do a good initial job: https://gist.github.com/simonw/8174f653a0c08c120830c56332054...
So are you. https://ceoln.wordpress.com/2023/03/31/its-just-predicting-t...
Counterpoint: https://the-decoder.com/why-large-ai-language-models-dont-le...
Qualifications are a great way of figuring out who's worth listening to, but that's not always the case as AI/ML folks prove with regularity, but it's nice to know if someone isn't just an armchair commentator and isn't peddling baseless information.
That's not what it's claiming at all. It's taking umbrage with the word "just" in "just predicting the next word".
A gun "just moves some metal from one place to another".
I'd sooner listen to people who are leaders in their field when it comes to LLMs than someone rambling on their free WordPress blog.
It is just an LLM. That's an objective fact. IBM's Watson is a pipeline of different algorithms. Neither are like a human brain, but multiple processes are going to be more brain-like in concept than just an LLM.
Followed by the link. You made the claim that I'm an LLM.
What actually is your point?
Tbh, it isn't surprising that it fails, its surprising that it has subjects where it is very useful.
My team has been working on several advanced techniques using reasoning on LLMs. The stacked performance of all of these techniques combined yields is quite impressive.