I have had to make these choice(s) before. Here are some of my thoughts:

Do what make you happy. A caustic workplace is not worth it. The levels at work to me are meaningless. Don't worry what other people think.

Your health (physical and mental) & happiness is priority 1. You can always save money and work with it intelligently - if you need help -- get an investment advisor.

Stress and being sedentary are two different things. Getting out of being sedentary does reduce the stress. Walking is a great way to start and it is often underrated. Have the implicit belief that it does.

Stress and anxiety is complicated. Different people (genetics etc.) react to stress and pressure in different ways. I would seek out a therapist who can help you on this.

Remember -- we are in a pandemic -- so it all adds up