It's an irrational "feel safer" though. Despite no legislated requirement for mixed size impact tests, enough have been done to see that tiny cars do surprisingly well. Whether the microscopic Smart Car against an max size old Volvo estate, or super mini against a Range Rover. They are categorically NOT destroyed in an impact with the SUV barely taking a scratch.
I mostly agree with the rest though. :) My solution would be to copy the Scandinavians with a yearly car tax derived from a combination of weight over say 500kg and emissions.