The bulk of the problem is 5000lb+ vehicles. F150, SUVs like Suburbans, etc. etc. I'm not even sure if people are using these big vehicles to carry their families around, its probably just single-person drivers in most cases.
I'm not even sure how to approach the weight problem. We have plenty of cars in the 3000lb or 4000lb range but more-and-more people are pushing 5000lb+.
https://en.wikipedia.org/wiki/Fourth_power_law
There are some horrible states without annual inspections, but they should be federally punished until they comply. If we could force every state to have a drinking age of 21, using access to highway funding, then there's no reason the same thing couldn't be done for vehicular taxation. It would increase state tax revenue anyway...
I'm angry about this often, as mine is over fifteen years old. If I had to replace it with a new car, I'd want another Fit. The closest thing on the market is probably a hatchback Civic...and the whole reason for the Fit existing is because international pressure made the Civic too large and heavy for the desirable tax bracket in the Japanese market.
And it's not even small for the Japanese market, because they have the whole kei class, with strong incentives to drive smaller cars. Which is how it should work everywhere: severe tax penalties for larger cars, to promote public safety and fairly pay for the fourth-power law that affects road wear.