> Type safety is not factual correctness.
I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
I very much agree with this and want to hone in on where do actually disagree. Would you say a linear classifier hallucinates?
You’ve already agreed that this doesn’t establish correctness. An approve for an unauthorized action still meets the schema guarantee.
That’s why I find the messaging misleading. You’re acknowledging the limitations in these replies while defending the broader reliability pitch.
Even granting that each answer is calibrated individually, that doesn’t establish calibration of the decision that combines them.
Sure, I can threshold a composite score, but there may be many wrong answers with the same score. An unauthorized action doesn’t become acceptable because it scores highly on the other dimensions.
I still have to define the constraints and test which wrong actions get through the complete workflow on my own data. That’s a substantial part of the work being pushed back onto the developer.
From the intro blog
"Hallucination and type-safety are intrinsically related"
I'm not entirely sure why we're conflating type safety with, I guess, value or output safety.
"Would you say a linear classifier hallucinates?"
No, but it can be (and often is) mathematically correct and functionally incorrect. It doesn't help to say "a linear classifier can't hallucinate" when you get even 99% accuracy. That's 100% a semantic play, and it doesn't help when the picture of a dog is labeled cat and the response is "yeah but that's not a hallucination, only stupid LLMs do that"
IMO your system can make mistakes that are similar in spirit to hallucination (i.e. answering with a false answer instead of abstaining to answer).
Model input: "Does the user want to speak to a human support agent?"
Output: Yes.
I imagine that your model would produce this, and I think it's fair to say this is a hallucination. A human would caveat it with: "Yes, but not right now.", your model is incapable of that. Yes is technically correct, but within the context of being in a live chat, a human would understand that the caveat is required.
Your question would correctly classify the user's input as requesting a human support agent, but at an indeterminate time.
If you wanted to determine whether the user wants to speak to a human support agent immediately, you would have to correctly qualify your question, e.g. "Does the user want to speak to a human support agent now?". You could have another question which is "Is the user requesting a call-back from a human support agent?". Or you could have a multiple choice query which would filter the conversation into one of a number of pre-written possibilities.
This is nothing to do with accuracy or hallucination. It's a different method of interacting with the model where you are relied upon to be precise.
"call_back_day": {
"criteria": {
"none": "The user does not want a call back.",
"today": "The user wants a call back today.",
"tomorrow": "The user wants a call back tomorrow."
},
"instructions": "When does the user want a call back?",
"type": "choice"
}
linear regressions hallucinate in the simpson's paradox.
the model output can be quite confident and not representative of reality
- Give Jev and LLM the same input
- Lock down both to approved/rejected/unknown (LLM restricts on decoding)
- Both can be wrong, but neither can hallucinate (invent an another option).