This is all true, but the "rainmaker" doesn't have to be the CEO, or just one person, or even work at the company. An alternative is to find an external "rainmaker," an example that I've often seen is small consulting firms that focus on things other than development (e.g. design or organization), or focus on a different type of development (e.g. CAD/CAE), that need reliable partners to bring onto their own projects.

This is probably less good long term, but it depends on what you're trying to do as a development shop, and doesn't necessitate career changes.