You can definitely prevent a lot of file/executable accesses via SELinux by running sshd in the default sshd_t or even customizing your own sshd domain and preventing sshd from being able to run binaries in its own domain without a transition. What you cannot prevent though is certain things that sshd _requires_ to function like certain capabilities and networking access.

by default sshd has access to all files in /home/$user/.ssh/, but that could be prevented by giving private keys a new unique file context, etc.

SELinux would not prevent all attacks, but it can mitigate quite a few as part of a larger security posture