First, driving in the UK is much more a privilege than a right as in the US. You can live a complete life in the UK without a license because of the wide availability of public transit. In the US however, if you want to maintain a steady job outside of NYC, Chicago, DC, Boston or perhaps a few others, you'll have to drive. Revoking a driver's license in the US can be life-altering in a way that it just won't in the UK. Fewer people bother getting the license and fewer still drive.
Second, driving is much more physically and mentally demanding in the UK. Perhaps that serves to reduce traffic deaths by forcing focus, but it also imposes a limit on the types of people who can drive here. This selects against too young, too old, too small, disabled, etc. in a way that would not be tolerated in the US for the above reasons.
Third, annual vehicle inspections are much more stringent in the UK which takes a lot of older vehicles off the road and again selects against those of lower socio-economic status in a way that would be unconscionable in the US.