Author of deeplearnjs here. We hear you, and we 100% agree. Stay tuned.
By the way, if you'd make your interface more general than deep learning, your library could be the start of an alternative for numpy/scipy on JS, and it would be even faster than the original Python version because it uses the GPU. Just a thought ...
(One small downside is that JS doesn't have the nice operator overloading that Python has, afaik)