If your job is to make an accurate parser for it, probably you want to hand code it. If your job is to make sense of the data the customer has provided you with, this is merely an impediment to your actual job, and ChatGPT has you covered. Yes, there'll be mistakes. But ChatGPT can do in a few seconds what'd take you hours.
Computers are very good at formal rigor, and we have quite some rigorous methods of program synthesis.
Whoever manages to connect these dots will own much of the industry.
For those who answer "yes" to the above I'd encourage them to read the story of the Therac-25 [1], a machine where hardware mechanisms in model A where replaced by software mechanisms in model B leading to a race condition that would dose patients with massive doses of radiation.
> Yes, there'll be mistakes.
"Over the following weeks the patient experienced paralysis of the left arm, nausea, vomiting, and ended up being hospitalized for radiation-induced myelitis of the spinal cord. His legs, mid-diaphragm and vocal cords ended up paralyzed. He also had recurrent herpes simplex skin infections. He died five months after the overdose." [1]
"But your honor, I specifically wrote 'add unit tests for important functionality'!"
That said, I don’t think it’s a strong rebuttal to the arguments here. Context matters. Earlier poster mentioned they’d use it for a React front end but not a backend deletion.
Knowing the tool and the appropriateness of the tool in context is one of the lessons to be learned from the Therac 25 tragedy.
Assurance is a complex topic, and any safety critical device should have a carefully thought through architecture and rigorous testing program which minimises the risk of incidents. It therefore seems scarcely relevant here, beyond the fact that a well defined delivery system should be able to handle multiple human errors during implementation without leading to crucial failure modes occuring.
The problem with using an ML model to parse stuff you don't understand is that you then have no way to verify the accuracy
If there are no stakes to the results then that's fine to trust blindly but if this is something you need for your job, that's risky and franky stupid to trust to ML
Lots of the time you understand the problem, but the problem is repetitive. Parsing a weird file format might well be that. Beyond that, you have solutions that are easily checked. For example, if I ask ChatGPT to optimise an algorithm for a certain CPU cache, I can easily read whether it did that. And then, there are parts of a software job that are crucial and subtle, and parts that are not.
As a practitioner, traditionally that leads to a shift in the focus and speed with which you approach a task - some pieces of code are 100 lines that took you 2 weeks to get to and were hard fought, some are 2000 lines which you wrote in a day.
Lastly, so much of solid software is being able to understand a probably unfamiliar domain, and ChatGPT can be a great buddy in terms of gaining problem context, finding the limits of your own understanding.
I don't use co-pilot like things, but I've found ChatGPT to be a massive enabler in terms of being able to be productive in unfamiliar problem-spaces.
I only use it sparingly though because it can too easily become a crutch where you start to feel a bit lost without it.
Edit: before someone accuses me of doing what trovalds is saying, I understand that code now. I was able to probe the model to explain it to me to the same degree as my understanding of my own code. It was the same feeling as when my team innovated over my previous work and I just asked them how they did it.
In both cases, even after back and forth with the AI for half an hour, the result is absolute garbage. And never works. I don't understand at all how people get even barely usable stuff out of AI...
(1) Using LLMs as a learning aid (or docs augmentation)
(2) Writing one-off _whatevers_
(3) Finding the appropriate docs/info in the first place
(4) As a meta-tool to sniff out whether other people have gone down this path before
(5) Cloning any sort of boilerplate "pattern" which requires more than simple text substitution (nvim macros, custom CLI tools applied to a visual selection, ... can get you a long ways, but they're not good for everything)
For a few select examples:
(1a) You can copy in a snippet of code and ask what _this_ character does _here_, finding appropriate search terms to actually learn about the topic.
(1b) Generated examples are fine, even if they're wrong, if you use them as a mental scaffolding to learn more about a library. Analyze them and prove them right or find why they're wrong, and that process of analysis teaches you in a different way than just reading the docs by themselves.
(1c) Sometimes the docs are just unwieldy. Ask the LLM for the snippet immediately after the "-B" flag, and then you can search for that more-unique snippet instead of, e.g., 1000 occurrences of other fields referencing "-B" and not defining it.
(2a) Say you want to know how your webserver responds when the client sends each byte in a separate syscall, waits a second between each one, and then immediately starts a new pipelined request doing the same shenanigans without waiting for a response to the first one. The LLM can generate exactly what you need, or when it fails it was still faster than typing it yourself. It's a task that's much more succinct to describe in English than code, yet simple enough that the LLM has a high success rate, and a brief scan of the code can confirm it's actually hitting localhost instead of downloading not-a-virus.exe.txt. The stakes are low if it's wrong because you can visually tell if it's doing the right thing.
(3a) I recalled reading once about the design of the voting/scoring algorithm used internally in Google's TGIF question ranker and knew there was a brief public blog post going over the more important details in a consumable format. Current search engines mostly can't point me to an old, unmonetized, niche blog post, especially when I don't remember any of the important keywords. LLMs can often point me to the exact blog, and barring that can usually give me the keywords I need to find it myself (similarly with finding things like the AWS SDK C API, which exists but isn't advertised and is hard to search for because the other keywords dominate). In this case, I wanted to read about Wilson's Scoring by Evan Miller [0].
(4a) Ask the same question a few different ways and a lot of times. If you only get garbage hallucinated blog posts, research articles, and keywords not related to your current task, it's suggestive of the specific task you're doing not being popular. If at a high level you know your end goal has been implemented many times, that's suggestive of you going down an incorrect path.
(5a) I was writing a program heavily leveraging lazy type instantiation to generate a zero-cost (de-virtualized, inlined where appropriate) iterator library. Anyone can write map and reduce, but it's a little more annoying to have a meta-layer doing that sort of thing in the type system, especially with some other constraints I had in the project. However, showing the LLM a few examples of this type-level way of defining the things I'm doing and then asking for implementations of `peek` or `skip` or whatever worked swimmingly. They're well suited to that task because the boilerplate isn't just simple textual substitution, because the tasks are simple enough that the LLM has a high chance of getting them right, and because the nature of the library is that a couple very simple hand-written tests will catch all the subtle mistakes, and a human reading the code will catch all the non-subtle errors.
And so on. I don't use LLMs for everything. If I want to do anything involving infrequently used Python dunders I'll probably go straight to the data model docs [1] and search within that page. I write most code by hand (give or take many custom tools I've integrated into my nvim workflow). Sometimes an LLM makes my life easier though.
[0] https://www.evanmiller.org/how-not-to-sort-by-average-rating...
> You might be surprised to learn that I actually think LLMs have the potential to be not only fun but genuinely useful. “Show me some bullshit that would be typical in this context” can be a genuinely helpful question to have answered, in code and in natural language — for brainstorming, for seeing common conventions in an unfamiliar context, for having something crappy to react to.
> Alas, that does not remotely resemble how people are pitching this technology.