I am a SWE and I work close to the metal. ChatGPT is helpful, but it always says I shouldn’t do what I’m doing and I should use safer abstractions for my work. I write the low level abstractions, that is my job. Even if I ask it not to, it soon (over a few dozen messages) forgets.
My friend is a doctor and it’s the same story. He is always being told to see a qualified doctor about his questions. It just comes off as patronizing.
Sam Altman said a while ago that he believes the future of LLMs involves the user being able to customize them. I hope he hasn’t changed his mind. I definitely want a model I can make just give me straight answers without the social responsibility/legal disclaimer attached to each.
Come to think of it… what happened to legal disclaimers buried deep in the EULA? Why are they burned into nearly every answer LLMs (including Claude and Llama2) give us?