Could a viable company be created with the goal of producing and selling an MMO engine, rather than an actual MMO?
You would be removing the expense and risk of creating the gameplay side of things (models, textures, quests etc), and could focus on getting the engine right. There would probably be some issues in selling the engine though: there would be a certain chicken-and-egg factor in selling an unproven engine to a new MMO-building company.