I think that is totally fine! Especially if it is organized that way from the start.

But as soon as you allow contributions, is the project really yours anymore? Is it fair to the community that has contributed to your project, all of a sudden to be locked out from it, and forced to fork it to continue it's evolution? Why not hand it over to the community and start your own fork for personal development? I think that is the larger debate to be had.