Hacker News Reader

Comment by fleitz | Hacker News Reader

Is it just me or would this not drastically speed up calculation of the weights for neural nets?

SVD is in fact used to compress dnns.

Of course, like many things in ML, as soon as you make it faster, you just make the model bigger and turn the speed gains into accuracy gains.

Well the Ax=b part is quite general as long as A has a special structure (diagonally dominant).

Good luck coding it though, I'm still looking for an implementation of a low stretch spanning tree (needed in order for the solver to be really lineaR)

"Using Petal-Decompositions to Build a Low Stretch Spanning Tree" (http://research.microsoft.com/pubs/163400/spanning-full1.pdf):

  Stretch: O(log n log log n)
  Time:    O(m log n log log n)

Could you please explain me how? I watched the lecture, but can't make connection. I understand how neural networks work and how they are optimized. I'm stupid. Thanks

Heat flow is akin to 'information' flow in a neural net because the weight of a node is the sum/average of the weights of all nodes it's connected to just like the heat of a node is the sum/average of the weights of all the nodes its connected to.

Reply on news.ycombinator.com