I work with phone systems. It's usually determined by how long they hear a continuous sound at the start of a call. If it's relatively short (and it can be adjusted by settings), it assumes it's a live voice (like someone saying "Hello?" is usually pretty short). If it's continuous for a certain period of time, it assumes it's a voicemail.
For our speech applications we have different behaviors depending on which call disposition is detected (that's the term for this, also determines things like invalid phone numbers).
Sometimes when testing speech applications I'll pick up and scratch the receiver continuously or speak without taking a pause until the voicemail disposition is triggered so I can test a voicemail message without having to check voicemail.
We make these calls on behalf of health insurance companies for health reminders and surveys, so our calls are more legitimate than most, but we do use an autodialer to reduce the load on our live agents.