Hacker News Reader

Comment by mikektung | Hacker News Reader

Low power and multicore are cute and all, but imagine the type of machine learning we could do on 384GHz cores.

Ok, now imagine what a 384GHz core would cost.

(hint: probably more than the world GDP. Each)

Ok, NOW imagine what 10,000 3.84GHz cores would cost (100 times more aggregate cycles per second than 1x 384GHz core). What's that, you figure a measly $10-30M instead of more money than exists on earth?

Any research simulation is going to want to be parallelized anyway. You'll bump into the limits of the 384GHz core, no doubt about that, at which point you are back to distributed computing. For limitless complexity and limitless appetite for computing power, distributed computing will always be the answer.

Imagine the type of machine learning we could do on low power 384GHz multi-cores

Imagine the type of machine learning we could do on low power 384GHz quantum multi-cores with plasma exhaust relays and phase lock inverters.

Invert the polarity

Imagine the type of machine learning we could do on 384 low power 1GHz cores.

Not everything parallelizes nicely. Given enough processors, serial computation eventually becomes the bottleneck:

http://en.wikipedia.org/wiki/Amdahl's_law

Many machine learning methods parallelize extremely nicely.

I'm afraid the machine you'd get would be a furnace or a torch... AFAIK power that you'd need to dissipate is linear on frequency, so unless we will discover technology to radically lower the voltage used in CPUs or some other parameter, dissipating 100x heat would be kind of a big problem for such CPU.

In one clock cycle, light would be able to travel about 780 micrometers. The real propagation speed of signals on a chip will be substantially less than the speed of light, and it gets slower at smaller feature sizes. This could be tricky.

http://en.wikipedia.org/wiki/Interconnect_bottleneck

Imagine a Beowulf cluster of these!!!

Unfortunately, many machine learning algorithms aren't easily parallelizable. Consider any boosting ensemble learner, such as the popular random forest, which optimizes the result from the previous iteration. (I run the machine learning startup diffbot.com and we could really use faster cycles.)

what sort of algorithms would you like to have faster variants/implementations of?

You could invent 1000 new ML algorithms in the time it would take you to engineer that.

The machine learning I did was never limited by CPU power, but rather my algorithm and especially the states. Asking for better performance without better states is garbage-in-glory-out.

Reply on news.ycombinator.com