If you have a completely remote team, you're paying at least something in productivity, because training people to be open to hop on a VC or chat at a moment's notice and documenting everything through email or a shared document is not trivial, especially when you're not used to doing things that way, and the costs are not zero.
Or you could do everything asynchronously to deal with time zones, but then you're introducing an inevitable lag based on overlap of working hours.
For a healthy, functioning organization, it might be worth it. However, if your team is already dysfunctional, having remote workers exacerbates the problem. (I have first hand experience)