FreeSWITCH can do this in mod_conference using OpenAL, its pretty sweet.