That’s fine, GCC on sparc will generate the sequence to do an unaligned read with the right endianess. If that means reading bytes and shifting them, so be it.
If you know you are reading aligned memory and you need to do it with a single word memory access, then you need to use intNN_t* and use htonl or similar functions to byteswap. There’s no way around it.