No one is yet sure whether traffic will increase or decrease.
As you describe, people may buy self driving cars and use the self-driving feature to drive them more often. Wealthy folks and people in rural areas are almost certain to take this route.
Others predict that people won't buy cars at all and will instead rent them from ride services, who in turn may promote ride sharing on an unprecedented scale. Poor folks, for whom a car is a major expense, are almost certain to take this route, as well as city dwellers.
Personally, I think we may see traffic increase in some areas and substantially decrease in others. I suspect the largest source of traffic -- the commute -- will shrink considerably as networks of reasonably dense self-driving vanpools and small busses pop up.