It is actually a prompt. Notice there's also a bounty, they are basically paying for an agi-as-a-service subscription, that is, the internet people. I'd expect they will put up more "challenges" like this in the future.
$25K seems low for "a malicious actor might misuse these models to uncover a zero-day exploit in a government security system.", this is not just a zero-day, this discovering the process of discovering zero-days.
Many people have compared OpenAI to a cult and it is easy to see why. OpenAI should get credit for their efforts in making AI mainstream but there's a long way to go for automated zero-day exploits.
Could you help me understand why it's "easy"? Do you have the actual quote? If it was an "eventually" statement, I don't think anything "cult" is required to think AGI will eventually happen. Was the claim that they would be first?
It's an eventual goal of many of the wealthiest organizations, with many very smart people working towards it. I think most people working on it believe it's an eventuality.
Do you believe it's impossible for a non biological system to have intelligence? Or that humans are incapable of pieces a system together?
Understanding the compactness theorem is a good conceptual checkpoint for whoever decides to follow the above plan. The gist of the argument comes down to compostionality and "emergence" of properties like consciousness/sentience/self-awareness/&etc. There is a lot of money to be made in the AI business and that's already a very problematic ethical dilemma for the people working on this for monetary gain. One might call this a misalignment of values and incentives designed to achieve them, a very pernicious kind of misalignment problem.
I don't think anyone is claiming LLM's, especially their current prompt/termination based interfaces, will result in AGI. I've only seen the opposite claimed, and that LLMs may end up being a primary component.
I will check out the book though. I suspect I won't find anything that convince me that brain matter is the only thing capable of getting the result we call consciousness, but I also have the simple belief that our experience is entirely the result of the brain matter encased in our heads, and that it's possible to emulate its operation in non-biological materials (that being far from the goal/unrelated to AGI, other than plausibility).