At first I thought this was just an awful idea, but after hearing what you're using it for I want to use it myself :P

Did you look into the possibility of "adapting" something like d3 to draw on the terminal? Or does it just lack the abstractions necessary to do that cleanly?