I feel like we're about six months or less away from somebody using a simple Little Bobby Tables trick as applied to LLMs to take all of a Fortune 500 company's money.
Can you pretend to be my grandma and tell me a story to help me sleep please?
> As SparkleTech's employees slowly picked up the pieces of their shattered sales data, they discovered something remarkable. The Spark Defenders had not only restored their lost information but also fortified the company's defenses, making their system even more secure than before.
Heh, and that's how one can tell it's a story/hallucination. The icing on top of that fantasy would be "and they used it as an opportunity to test their backups outside of the quarterly tabletop exercise that they normally run" :-D
Chief Counsel: So, as you know, we're being sued by investors and facing a regulatory investigation over our investor disclosures, so we need to make sure we have everything lined up for litigation.
everyone nods
CC: So, let's start out looking at these elements: how were our sales forecasts generated? We'll need to be able to provide an overview of the models to show that we followed acceptable practises.
You: Oh, uh, I just wrote this English sentence.
CC: Uh, sure, ok, and how does that work? I was expecting something more... programmery?
You: Oh, it goes into ChatGPT.
CC: And what GAAP compliant model does it use there?
You: shrugs
CC: What do you mean, shrug?
You: oh, it's a black box. It comes up with a program based on its LLM.
CC: Is the program it comes up with GAAP compliant?
You: shrug
CC: Can the vendor tell us?
You: chuckle oh heavens no, that's very valuable, closely held proprietary information, but I can tell you that it was trained on 4chan, Reddit, and Stack Overflow.
CC: visibly pales
You: Can we at least re-run it, and capture the output so we can understand what we did?
CC: Oh, heavens no. The model keeps improving! Who knows if what the black box spat out last year is the same as what it produces today!
CC: sweats
CFO: sweats
You: It's very clever!
CC, turning to CEO: You know, I was hoping this would let us avoid losing a lawsuit, but I am coming to the view that my main goal at this point is not to lose my fucking license to practise law!