Routing belongs on a mobile device. Now all I need is a usable Android app for browsing maps and navigating. OsmAnd seems to be the benchmark, and it's freaking terrible.
For your usecase, have a look at mapswithme instead. It's proprietary but very neat. Does only the basics but does them well.
There are plenty of mobile apps using OSM data, some of them high quality and improving regularly. If one app doesn't suit you, look around. Have another look if you last tried many months ago.
It's a classic open source problem: pack in the functionality, but not be entirely sure how best to present it, so provide everything with lots and lots of switches for power users to toggle what they want.
Starting out with less stuff might help; starting out remembering to go to the map would, I suspect, be a great UX experience for a map app.