I'm not saying by default. I'm saying have it on the CD.
(It would be nice to have a one-click tool that builds you a customized image with your own SSH public keys baked in. Ubuntu doesn't have to run this tool - actually there's probably a cool project idea for someone in standing up a little website to do this, either by letting you paste in public keys or grabbing them from the GitHub public API.)