That depends upon the type of SSD. There’s compact flash which is natively IDE, and the controllers are extremely weak, then there are modern TLC SATA drives where the controller would probably be a bit more powerful than the machine’s CPU.

It wasn't out of the ordinary at the time. See C64 floppy drive CPU.