Once you get beyond the basic crud-webapp-twitter-clone the ability for a new person to simply pick up and start working becomes increasingly more difficult. This is a cost that you will pay. You'll pay for it when one of the three people that understand the system decide to quit or when you have to put new hires through eight months of training before they can write a single line of code.
I've worked with systems that were written by people that have decades of industry knowledge. Every class, method and comment is using their lexicon and entire chunks of the system rely on assumed knowledge. Suddenly ranked_pct(std_dev_norm(userid,norm,v,arr,false,get_data(userid))) is returning 0.24 when it used to return 0.51 and the BA/PM would like to know what changed. That's when you'll realize that your growth has been severely limited.