Deeplearn.js is designed for webGL on a browser. I'm not sure if it'll work in Node. And on a server you'd probably want to target OpenCL.

Node adds file support, but libraries for stuff like dataframes are still in their early days. That might not be a problem for example problems, but real world data needs cleaning up before you can start doing ML.