Living on a fighter base, I met two people who were forced to eject from a plane. One lost 2" of height due to severe back/spine compression, one bounced off the ground so hard he could never fly again due to leg and hip issues.

Not to mention, an ejected plane can now go hit some school or office building, whatever causing all sort of casualties. Most pilots would rather just try and survive the crash and risk their own life.