It often revolves around whether the company has a physical place for you to work. Just because you are not using it may not alter the tax situation.
The issue with the current situation is that in principle I could be doing the work in New York (it doesn’t matter where I do it), but the office is closed and I don’t live in New York. So I have strong reasons for not doing my work in New York, but in principle I could be doing it there.
I guess there’s a further clause that you have to spend at least one day out of the year in New York to be taxed there, which unfortunately I have done (despite no longer traveling there for any work-related purpose).