I've been living alone (with dog) in a 540 sq ft garage conversion for 7 years and it's just about the perfect size for me. It has a small bedroom (queen bed), larger office, even larger LR, smallish kitchen and a bathroom (no tub). If I bothered to change antying, the kitchen could be slightly larger, the bathroom could use a tub, the laundry could be moved indoors (it's out on the covered deck) and the water heater could be moved indoors too. It's not well designed and space is not utilized well (bare walls could be shelves). But with a better design I think it would be completely satisfactory. I like having less to clean and less to heat, but I wouldn't attempt anything smaller.
I should add that my house has nothing to do with affordable dense city housing, it's on a farm in New Zealand, which has other buildings (garage, storage container, shearing shed).