Is that all that Mythos did?
Did it find any real potential issue, optimization/simplification opportunities, or sparked any thought-provoking discussion within your organization?
Or was it purely a net negative experience?
Did it find any real potential issue, optimization/simplification opportunities, or sparked any thought-provoking discussion within your organization?
Or was it purely a net negative experience?
You're the only one coming away thinking there was a net negative experience.
The only thought-ptovoking discussion should be "why the hell do we have this stochastic parrot anywhere near out codebase"
Not saying that is the situation, I don’t know. But if “one error is too many” is your point of view… do you think the humans in these orgs are 100% perfect 100% of the time?
How many gaps have humans not caught?
> But if “one error is too many” is your point of view
Yes, in regulated industries "one error is too many" is the only right approach.
Yes, humans also make errors, and there you have a range of options: from tracing and finding the causes of the error (and tightening processes) to literally jailing those responsible. Your hallucination machine will happily "identify" 17 gaps, and create 34 more. And no, there are no processes to make it better. The "make no mistakes" incantation will happily be ignored for obvious reasons, regardless of how many forms of it you throw at it.
You're basically saying "we need human review for literally everything AI outputs because we have no way of saying whether anything it produces is hallucination or not. And since it produces plausible-sounding things really fast, it puts enormous burden on human reviewers".
The machine doesn't say that. It says "Here are 170 completely correct and verified results".
You have to check and verify all of those results yourself, and on any given day it can be anywhere from 0% to 100% incorrect.
> I assume you've come to this conclusion based on some reasoning, but you're not sharing it in this response AFAICT.
The reasoning comes from actually working with AI tools. And the reasoning can be seen in the actual comment this tgread started from: https://news.ycombinator.com/item?id=48434824
>In a regulated industry 90% false positive rate is indistinguishable from 100% failure rate
So defending that position on the basis of it not actually being a 90% failure rate would mean you shouldn't have taken it in the first place. The fact that the LLM lies about its failure rate is nearly irrelevant; the machine could output the literal string "The following is 90% likely to be a false positive: " followed by the LLM output.
The reasoning in the comment that started the thread is "it's annoying to redo human review". Your position as I understand it is that there is no or negative business value to a tool that spit out a list of potential issues of which 10% are real issues with your business. This is what I fail to understand. This would be an incredibly useful first step towards any audit and would save loads of money. Why not?
A system which will just randomly decide to give the legal team reasons to not back you up is:
* A system whose output will get brought up in lawsuits and make legal's job harder.
* A system that will make the dev team perpetually chase its tail while it oscillates between the several different valid interpretations of the rules.