What you describe will be safe when all 20 of those cars are self driving. In the meantime, your trying to get 20 people to agree on what is the correct speed. Not only that, but your trying to get 20 people to hold that speed exactly. Invariably what happens is that a couple cars are going to go a little slower up hill (or whatever) and cause yo-yo'ing in the line, which is what causes actual traffic jams.
The result is the guy in the back is going to be speeding up and slowing down around the speed limit. That is extremely annoying. If you haven't been this guy, I can only assume your always the guy in the front cruising in the fast lane causing 19+ people behind you to get angry and want to kill you.
Basically, travel in the right lane, unless your going to put the effort in and pass the person in the right lane within a reasonable time-frame (say in under a minute). Otherwise your just creating more problems. Wishing that isn't true doesn't make the problem go away, its just creating unsafe conditions for other people, who are frequently just trying to drive as safe as possible too.