IMHO it is not hard or difficult per se, but you need to design your company around it, like with other things.
It is also an area that needs more study in order to develop fully.
Probably if I had a conventional company and I had to transform or convert it to remote, it wouldn't work. There would be vested interest against the conversion on some people, some people would not believe it is not possible(people tend to believe that if you don't look busy to others you are not working) or managers will fear their employees slacking.
When you create a company that makes money from day one remotely, you have not this problem, as you organize around it and it is not optional.
In some ways it is weird. If you go out of your house to get your children from school because you did not spend 1 hour commuting to work, 1 returning from it, society could believe you are not working at all.
They are often surprised to see you have money, and much more money than they have.
But I suppose it is not different from an old farmer watching people sitting in a chair in the city call it "work".