Perhaps this has changed recently, but the last time I tried to use Uber without "allow location access even when not using the app," I was unable to call a ride. Instead, I was given instructions on how to enable that setting.
I've used it constantly without location services since the day they first requested the feature. When I open it, the splash screen has 2 options - Enable Location Services and Enter Pickup Address. Perhaps you missed the second option?