I've noticed your implementation uses websockets, which are built on top of TCP. we used UDP and encapsulated everything is a pretty simple to use (and easily the most well documented part of the project) p2plib.c. the fear was that video and audio can get backed up if we were to use TCP. so audio and video were sent via UDP packets and rendered every time a UDP packet was recieved.
https://github.com/dhotson/txtcam
Another thing I'd like to try to get a bit more resolution is to perhaps use the braille character set in combination with 256 color mode: https://github.com/asciimoo/drawille
Also, some terminals are starting to support 24bit color (iTerm2 nightlies) which could pretty drastically improve the possibilities of terminal based video: https://github.com/frytaz/txtcam/tree/color :)
Do you have any good examples of calculating the right foreground/background colors to use?
http://aa-project.sourceforge.net/aalib/
http://caca.zoy.org/wiki/libcaca
and possibly other clones too. Might help improve the visuals.