I'm not sure my simplified math is valid, but for example the F-150 frame with Cd = 0.36 and 2.9m² frontal area vs. the Prius with Cd = 0.25 and 2.17m² frontal area would use theoretical ~55 Wh/Km more at constant 80 Km/h (~50 mph) just due to air resistance (excluding ground friction etc.). Assuming a total efficiency of 20% of a gasoline engine and 1 litre of gasoline containing 8.8 KWh of energy, this means approx. 3 litres per 100 Km more gasoline, just due to aerodynamics (not weight etc.).
For a less extreme comparison, the (2006) Mini Cooper S vs. the Prius I get 13.45 Wh/Km difference, meaning 0.7 litres per 100 Km (at constant 80 Km/h, only due to air resistance).
Perhaps one day it'll be possible to attach the self-driving car to the back of the car in front on boring commutes to get rid of this factor. ;-)