That's must be annoying to say the least. In India drivers require an OTP to start a ride.
The OTP is the same for a user across rides, so I have mine memorised which is nifty. No fiddling with the phone during boarding.
On security: exploiting this would require the driver to stay in my vicinity the next time I book a ride, and also get the ride assigned to them. In a high population density area, it's rare - I've never had the same driver twice.