No, people who you don't follow can ping you saying they've sent you a message. Your software can simply drop them on the floor if you don't care to receive messages from people who you aren't following (AKA people who aren't on your whitelist.)

No polling is involved.