Health care seems to be a big one. Few HR departments are frequently equipped to provide another kind of health-care package, and health care expensive, heavily subsidized by many tech companies, negotiated because of bulk sales, etc.
Depending on the business, the company might actually be subject to massive sales taxes as well if they employ someone out of state. Some states, like CA, have enacted rather byzantine sales tax laws, such that if you were, say, a company in Idaho, and someone wanted to move to CA, you'd all of a sudden have a "presence" in the state and be subject to more taxes.
These are just off the top of my head. So basically, unless the CEO wants it prioritized, I suspect the "finance guys" are actually just identifying real costs. So I wouldn't expect remote work to just jump wholesale nationwide without some real infrastructure in place. In any case, the silicon valley area sure seems to have some pretty stark incentives for prioritizing remote work. I'd expect the issues with remote work to be ironed out faster then say, a massive wage raise or controlled real estate costs.