mpv has a true color terminal renderer.[1] It's not the fastest though: http://i.imgur.com/919Rez5.png

  $ mpv --vo tct https://www.youtube.com/watch\?v\=PA4InNa0NSQ

[1]: https://mpv.io/manual/master/#video-output-drivers-tct