We're a two person team and we've been doing it for the past 5 years like a clockwork. On of our recent sites makes around $200K/year so what you're after is definitely doable.
You just need to understand two things:
1. You can't be the entrepreneur, technician and manager. You need to create an organization first and define the roles clearly on paper.
2. You need a system to do things. Even the smallest of the things need to be part of a system. Systems helps you improve what is working and discard what isn't. What this means to you is that is helps you to repeat your success over and over until you make it big while preventing you from making the mistakes from the past.
It's easier said than done but these are the only two things you need for your goal.
Bonus tip: If you find my advise useful, learn a little about BPM (business process modelling). It help you organize and create systems faster (though I've learned everything the hard way).