There's lots of reasons for congestion existing.

https://www.geotab.com/blog/traffic-congestion/

Given the same number of cars on the road, a self-driving fleet would eliminate many mechanical failures and many human failures, and be able to adapt to sub-optimal infrastructure at a network-wide level.

Infrastructure would also be easier or more efficient to improve, because you'd have removed much of the human variability that makes identifying choke points difficult.