The old problem of induction. It's interesting how many of the AI accelerationist crowd are (purposefully pretending to be) unaware of it.
The problem is, the POI applies to literally everything, including events which seem to have well-understood and well-established causes. That makes it impossible to know when it's going to be applicable in the real world and when it's not.