Using prediction error as the definition of curiosity rings hollow for me. Curiosity in my mind is more about mapping out an unexplored thing and not about being surprised.
If you're not surprised at any point in mapping out an unexplored thing, in what sense is it unexplored?
There are pretty high odds you've never been to this exact page before: https://oeis.org/A000079. But once you click on it, is there any remaining curiosity? It's an unexplored thing, in the sense that (I presume) you've never looked at this exact page before. But it doesn't provoke curiousity because there's nothing there to surprise you.
I'm trying to imagine the simplest case, say a button you could press and every time you pressed it something entirely random would happen, always guaranteeing surprise. It would have a great deal of novelty at first but after a while it would cease to hold your attention even though your prediction of what would happen would never be accurate. I'd bet that after a while you might even never bother to push it again. The only way you would be convinced to push it consistently would be a) if you were assigned a reward for pushing it e.g. money in which case it is a slot machine or b) if by pushing it you could somehow reduce your uncertainty about what would happen which would as a by product reduce your surprise.
Thinking about it this way, surprise is certainly a key element at first. It grabs your attention initially but it doesn't hold it. What keeps you focused on exploring the thing that surprised you initially is the learning process which involves reducing prediction error i.e. reducing surprise. So there is a tension between the two.
The combination probably makes for a good exploration strategy. Initial surprise, look for a learnable pattern and follow it until another surprise, maybe backtrack and try other familiar patterns until those are exhausted and then investigate each sequence that led to a surprise by recursing through these steps.
This would also explain the example where my curiosity was prompted by the unknown link but I was not motivated to explore further. The website wasn't interesting to me because it was too unfamiliar and I wasn't able to find any familiar routes to explore through it due to my lack of interest in that area of mathematics but our hypothetical mathematician with a fondness for integers would see lots of familiar patterns they could explore attached to which are likely some enticingly unknown and surprising links.
Thanks for the prompt to think about this more!
Literally getting dopamine rewards for seeing something new is what keeps people glued to tik tok feeds and twitter.
I tend to get bored halfway through a book if it is predictable.