Yeah, but lots of SoCs have special boot and SoC management cores with a different ISA. The Tegra X1 in the switch uses an ARM7, CortexMs are in vogue in a lot of designs, AMD and Intel have the PSP and ME respectively, and even the SiFive SoCs have a E core that boots the main core complex of U cores.
In none of these cases (including the RPi) is this core the GPU.