It sounds like your process isn't handling them well, or you just aren't making them a real part of your team.

At my company, we have half our staff in Melbourne, Australia... and half spread out globally around many different time zones.

The remote workers are every bit a part of the team as the Melbourne workers.

I think adopting a more asynchronous workflow makes sense, both to include remote workers but also it can be very efficient (remove distractions).

A tool like Campfire to handle communication is really important too, anyone can log in and see what they missed and you don't need to be there at the same time to have a conversation.

Also, you need to trust them and give a lot of autonomy, it can't be managed in a traditional way.