I see you used X. From what I remember when I looked at this years ago, X was really bad at highlighting dirty pixels. It couldn't tell you where your pixels had changed. So you had to flush the entire screen out over the network (vnc) every time something changed. This inefficiency made it unworkable in some cases.