I believe you're describing F-Droid.
Not using F-Droid wouldn't be better, it is just that it really seams hopeless for a complete solution without a major movement like what spawned GNU-Linux. All these problems need way more devs involved.
The system was generally unstable. I finally gave up when I noticed video playback in landscape mode turns the screen green, but works fine in vertical (Different apps, even ASOP stock apps, even after fresh re-compile)
I'm now going the other route... using a sony image and rip out everything unnecessary. Neither seems worth the time for average consumers. Learned a lot, though.