To my surprise they were all cat5 cables. With the house being built in 2003 this was surprisingly forward-looking.
I capped all the cables that were on the punchdown panel and put a switch in there instead, and replaced all the wall jacks with RJ45, and bam, working gigabit around the house, including PoE for my WiFi access points. Still haven’t had to punch any holes in the walls.