It’s like any other system. A middle man has to take their cut and will increase costs. There is no reason (that I know of) to think the dealership model would ever decrease costs.
I can think of a handful of questions that need to be answered besides just saying "middle man bad".
What's the cost of delivering dozens of vehicles from the plant to a dealership vs delivering one off vehicles to a house? Or the cost of an OEM having their own employees distributed to answer questions on a vehicle. GM has more than 4000 dealerships...would sales decrease because it would be less accessible to customers to see, test drive and service vehicles?
I don't believe the current dealership model of huge lots, tens of employees is efficient. But I'm not so certain an OEM handling everything themselves is optimal.