36 karma · joined November 8, 2014
I'm not sure it's quite that simple.
I have a couple of problems I can't solve myself, although variants have been solved by others, so we know they're human-solvable. Verifying a candidate solution, however, is relatively easy.
For example, I've repeatedly asked frontier models (including Fable) to produce a fast squaring routine for arbitrary precision integers that beats GWNUM running under Rosetta 2 on Apple Silicon. None have come remotely close to the best human-produced implementation. After hours of iteration, Fable's best attempt was still about 4x slower than the Rosetta-translated GWNUM, despite GWNUM not even running natively.
The point is that direct understanding isn't the only way to judge correctness. We often build tests, benchmarks, and oracles that let us validate artefacts produced by people or systems that are operating beyond our own cognitive abilities. Perhaps the real limit isn't a discernment horizon, but a horizon for constructing effective evaluators. Can we layer evaluators to get even more magnification; I'm betting we can.
So chmod +x file didn't work, now try python -c "import os; os.chmod('file',744)"
Not true, we do this because the 99% of the time it's true, however there are people who would be perfectly competent and responsible to drive without living to the age of 16-18. Same with voting, there are humans who have a deep understanding and intelligence about politics at a younger age than suffrage. Equally there are people who will be reckless drivers at 40 and vote on whim at 60.
We have these rules not because sophistication only comes through lived experience, we have them because it's strongly correlated and covers of most error cases.
To take this to AI, run the model enough times with a higher enough temperature, then perhaps it can solve your challenges with a high enough quality - just a thought.
Then later if it goes off piste in another session tell it to re-read the ADDs for x, y and z.
If someone could make that process less clunky, that would be great. However it's very much not just funnel every turd uttered in the prompt onto a git branch and trying a chug the lot down every session.
ESB simply add more complexity in the middle that needs yet another team to manage.
Long an short of it is the ESB is an anti-pattern, always has been and SOA and microservices are the same thing.