On page 50 describing the GPU instruction set, you should note right at the top the instruction LOADP which I quote is a:
"64-bit memory read. The source register contains a 32-bit byte address, which must be phrase aligned. The destination register will have the low long-word loaded into it, the high long-word is available in the high-half register. This applies to external memory only."
Not surprisingly, the companion instruction STOREP is described on page 56.
Now let's move on to the the Blitter on page 64: "The tour de force of the Blitter is its ability to generate Gouraud shaded polygons, using Z-buffering, in sixteen bit pixel mode. A lot of the logic in the Blitter is devoted to its ability to create these pixels four at a time, and to write them at a rate limited only by the bus bandwidth"
which is also a 64-bit operation.
Finally, while conceptually one could argue that the Atari 800 had a GPU (if you consider 4 players and a missile to be such) did the CD32 have direct hardware-assisted support for gouraud shading, texture-mapping (and if you were remotely clever, 64-bit texture mapping), and Z-buffering as the Atari Jaguar did? I don't think so but I'm open to being corrected here. I do think that having such support supports the claim that said Atari Jaguar was the first console with a true GPU.