There is no way every stop costs you 5 minutes, more like 1 minute. I am not a fan of buses because they are in the same traffic as cars plus they have to stop as you say. There are some mitigations like dedicated bus lanes that can swing the speed equation back in favour of buses. What you really want, however, is feeder buses taking people to the train, and it is easy to design a rail system that beats cars for speed in a city (basically any urban rail I've ever seen).