I mean it seems like you have a valid reason, but the raw cost in terms of just the loan payment isn't a trivial difference. "already over a third of my car payment" means your car payment alone is 3x the total cost otherwise, which is... a lot. Once you factor in maintenance, insurance, depreciation (which has been lower in recent years), it seems like it would be quite a leap if you didn't have a disability that made public transit more difficult.
I think the difference between transit methods within a city that has built-in alternative options has been interesting to examine now that I have a motor vehicle myself (that I got for free, but costs a lot to actually drive). I went without one for 8 years before someone offered it to me, and I do value the flexibility to occasionally get out of town wherever I want, but the tires alone will cost $2000 when I need to replace them, and the gas is crazy, so it's perfect in the sense that it sits there taking up street parking until I really need to get someone that a bike, scooter, bus, train, or walking doesn't work as well for. I'm not encouraged to use it for anything that it's really not needed for, and if it died tomorrow I wouldn't replace it with anything.
Relying on only the bus, even if the transit system is well-funded, has never been a particularly ideal situation, but the advantage is that I get that time back instead of being singularly occupied by driving. If there's traffic, I'd wildly rather be on the bus, but the train is always better.