Maybe someone put gum/something to block return coins. Then as people used the machine, the return coins built up. Usually the thief would return days later and remove the gum, along with the coins behind it.
Based on what others have said about how the coin mechanism works, this seems like the most likely explanation. Now it has me wondering if it was my brother who set it up. The payphone was outside a bar in an area I'd never been to before and I can't remember why we ended up there that day. Perhaps he was jamming up phones and then returning later to collect to proceeds. He was always getting into such small time schemes before he got mostly straightened out in the military.