Trolleys have a fixed route, which is not ideal. Fixed routes is why you see buses going down the road empty, or with one person on them.
The fixed-route nature of trolleys is not an inherently bad thing. In fact, it was the dominant (and eminently sustainable) form of urban expansion for over 75 years[1]. It turns out that fixed routes mean stability, which in turn means that businesses feel comfortable investing in long-term presence. Residents similarly find them desirable, since they can be certain that their streetcar won't be taken away from them (unlike a bus).
In my city, they haven't changed the bus routes in several decades.
Imagine if a bus had on demand stops, rather than stopping at 50 empty stops to get from point a to b. Those 49 extra stops are why nobody uses buses. It literally takes 4 flipping hours to go as what it takes 20 minutes by car, all while on a mostly empty bus, taking a fixed route, with fixed stops. Maybe it's better in other parts of the country, but in every city I've lived in, including San Jose, the nobody uses the buses, because nobody has that kind of time.
Trolleys for congested down town areas, sure.
But also: most municipalities that I know of do not require their buses to make stops if nobody is either waiting or signaling to get off the bus. NYC goes beyond that, and allows you to request a drop-off anywhere along the route during night service[1].
Maybe this behavour is very locale dependent?
That could be. Maybe they were ahead of schedule the few times I went, and the few times I observed them stopping at empty stops. This would make sense with the fact that they are almost always empty. They would come to a complete stop, open the door, immediately close the door, then peel away. It was comically discouraging. With the stop near my place, they would often just drive up, slow down to walking pace, then speed off.
Even if they were ahead of schedule, this would still mean the posted trip times were still many times greater than driving, which I assume is why they're empty.