What wifi dongle are you using?
* Panda PAU09 N600 Dual Band (higher power, 2.4GHz and 5GHz)
* Alfa AWUS052NH Dual-Band 2x 5dBi (high power, 2.4GHz and 5GHz, large, ugly)
* TP-Link N150 (works well, but not dual band)
It sounds like your realtek dongle may not support monitor mode.