The submissions can't use the internet. And I imagine can't be too huge - so you can't use "newest public LLMs" on this task.
We are also trialing a secondary leaderboard called ARC-AGI-Pub that imposes no limits or constraints. Not part of the prize today but could be in the future: https://arcprize.org/leaderboard