I'm with you. I'd like a fully free software stack across my personal devices, and I also thought that Ubuntu had the best chance of achieving this. I think the issue is in a business model that would work for Canonical.
OTOH, most of the hard work is done. My Ubuntu Phone mostly works. The stack is complete. I'd love to see interested people taking that code and keeping it going. I'd spend spare time on contributing if this were to happen.
(I'm a Canonical employee and Ubuntu developer, but not on the convergence/desktop/phone side, and my opinions here are my own and not that of my employer)