AI can't solve novel problems yet
jamesoclaire.com
jamesoclaire.com
I saw this hilarious exchange in one of the issues covering Apple’s recent changes that make it impossible to download IPA files.
Recreating an iPhone’s authentication to Apple App Store is a bit of a cat and mouse game. Apple recently changed it’s backend, so now dozens of projects are trying to find ways to circumvent or crack the Apple App Store download authentication.
Blacktop, out of likely boredom or curiosity, threw Claude at this issue for Claude to error out.
To be fair, it’s likely impossible for Claude to have figured this issue out. The ‘fix’ will be something quite novel, for example perhaps reverse engineering a way to authenticate with modern Apple App Store using an old Windows version of iTunes that is still supported.
Perhaps in the near future AI agents will be powerful enough to understand more complex environments, which will be interesting for these types of problems, but for now, it’s far from capable.
For anyone interested in the issue, this one is active as well: https://github.com/majd/ipatool/issues/357
Edit: You're right the 'problem' might not be new/novel, just that the solution is new.
I'm interested if anyone has an idea of what it would mean for an AI to generate or do something "novel" in the sense of creatively new.
There are 2 wolves and 2 sheep in a field. How many sheep will there be tomorrow?
Answer: There will still be 2 sheep tomorrow.
Needs some visualisation which LLMs don't really do well.