I think your numbers are off somewhere. Maybe you are thinking of some particular form of pollution rather than fuel efficiency? Short trips before the engine warms up are less fuel efficient, but nowhere near only 1/4 as efficient. Here's the US government suggesting a worst case combination of cold weather and a cold engine can cause a 24% drop in miles per gallon (ie, still more than 3/4 as efficient):
Fuel economy tests show that, in city driving, a conventional gasoline car's gas mileage is roughly 15% lower at 20°F than it would be at 77°F. It can drop as much as 24% for short (3- to 4-mile) trips.
https://www.fueleconomy.gov/feg/coldweather.shtml